Jul 01 19:19:11.491502 ip-10-0-139-28 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 19:19:11.491514 ip-10-0-139-28 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 19:19:11.491522 ip-10-0-139-28 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 19:19:11.491831 ip-10-0-139-28 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 19:19:21.652186 ip-10-0-139-28 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 01 19:19:21.652209 ip-10-0-139-28 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a448c1d3dd114c9797a21606111be501 -- Jul 01 19:21:27.761930 ip-10-0-139-28 systemd[1]: Starting Kubernetes Kubelet... Jul 01 19:21:28.193024 ip-10-0-139-28 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 19:21:28.193024 ip-10-0-139-28 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 19:21:28.193024 ip-10-0-139-28 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 19:21:28.193024 ip-10-0-139-28 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 19:21:28.193024 ip-10-0-139-28 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 19:21:28.194031 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.193944 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 19:21:28.199908 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199883 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:21:28.199908 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199905 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:21:28.199908 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199909 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:21:28.199908 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199912 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:21:28.199908 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199915 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:21:28.199908 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199918 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199921 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199923 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199926 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199929 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199931 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199934 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199937 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199939 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199942 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199945 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199947 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199950 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199952 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199955 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199957 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199960 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199962 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199965 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:21:28.200133 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199967 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199973 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199976 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199978 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199981 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199984 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199988 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199992 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199995 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.199998 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200000 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200003 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200006 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200008 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200012 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200015 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200018 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200022 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200025 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:21:28.200565 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200029 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200033 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200037 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200039 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200042 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200045 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200048 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200052 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200055 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200058 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200062 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200065 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200068 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200071 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200073 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200076 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200078 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200081 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200084 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200086 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:21:28.201028 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200089 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200092 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200094 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200097 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200100 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200103 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200105 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200108 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200111 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200114 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200116 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200120 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200123 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200125 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200128 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200130 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200133 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200136 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200138 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200141 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:21:28.201513 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200144 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200147 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200150 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200153 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200155 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200158 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200160 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200164 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200167 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200169 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200172 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200174 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200177 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200180 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200182 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200185 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200938 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200947 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200950 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200953 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:21:28.202004 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200956 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200959 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200962 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200965 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200968 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200971 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200975 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200978 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200980 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200983 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200986 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200989 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200992 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200995 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.200998 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201000 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201003 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201006 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201008 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201011 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:21:28.202512 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201014 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201016 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201018 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201021 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201023 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201028 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201031 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201034 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201038 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201041 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201044 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201046 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201049 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201052 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201056 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201058 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201061 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201064 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201066 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:21:28.203015 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201069 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201071 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201074 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201077 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201079 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201082 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201086 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201089 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201092 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201095 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201098 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201101 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201103 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201106 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201111 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201114 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201118 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201120 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201123 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201125 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:21:28.203467 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201128 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201131 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201134 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201137 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201139 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201142 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201144 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201147 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201149 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201152 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201154 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201157 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201160 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201162 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201165 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201167 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201170 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201172 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201175 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:21:28.203960 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201177 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201180 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201183 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201185 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201188 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201191 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201193 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201197 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201200 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201202 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201205 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201215 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201219 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201222 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201225 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201229 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201232 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201662 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201668 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201671 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:21:28.204416 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201674 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201677 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201680 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201683 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201685 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201688 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201691 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201694 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201696 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201699 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201702 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201704 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201707 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201710 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201714 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201717 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201720 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201724 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201726 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:21:28.204906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201730 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201733 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201736 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201738 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201741 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201743 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201746 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201749 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201752 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201754 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201757 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201760 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201762 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201765 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201767 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201770 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201773 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201775 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201778 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201780 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:21:28.205362 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201783 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201785 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201788 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201790 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201793 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201795 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201798 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201800 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201803 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201806 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201808 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201811 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201814 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201817 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201819 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201822 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201825 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201829 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201831 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:21:28.205854 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201834 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201836 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201840 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201842 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201845 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201847 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201850 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201852 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201855 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201857 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201860 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201862 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201864 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201867 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201870 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201872 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201874 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201877 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201879 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201882 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:21:28.206298 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201884 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201887 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201889 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201892 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201895 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201898 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201901 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201906 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201911 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201916 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201920 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201924 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201928 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201932 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201936 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201940 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201943 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.201945 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202008 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202016 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202022 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 19:21:28.206788 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202027 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202031 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202035 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202039 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202043 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202046 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202049 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202052 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202056 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202059 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202062 2570 flags.go:64] FLAG: --cgroup-root="" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202065 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202068 2570 flags.go:64] FLAG: --client-ca-file="" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202071 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202074 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202079 2570 flags.go:64] FLAG: --cluster-domain="" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202082 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202085 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202089 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202092 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202096 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202099 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202102 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 19:21:28.207312 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202106 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202109 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202112 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202114 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202118 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202121 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202125 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202128 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202131 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202133 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202136 2570 flags.go:64] FLAG: --enable-server="true" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202139 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202143 2570 flags.go:64] FLAG: --event-burst="100" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202146 2570 flags.go:64] FLAG: --event-qps="50" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202149 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202153 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202156 2570 flags.go:64] FLAG: --eviction-hard="" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202160 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202162 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202166 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202169 2570 flags.go:64] FLAG: --eviction-soft="" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202172 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202174 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202178 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202183 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 19:21:28.207856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202186 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202189 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202192 2570 flags.go:64] FLAG: --feature-gates="" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202196 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202199 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202202 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202205 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202208 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202211 2570 flags.go:64] FLAG: --help="false" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202214 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202217 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202220 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202223 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202226 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202229 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202232 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202235 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202238 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202241 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202243 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202246 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202249 2570 flags.go:64] FLAG: --kube-reserved="" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202252 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202255 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 19:21:28.208578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202258 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202261 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202264 2570 flags.go:64] FLAG: --lock-file="" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202266 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202269 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202272 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202277 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202283 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202287 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202290 2570 flags.go:64] FLAG: --logging-format="text" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202292 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202296 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202299 2570 flags.go:64] FLAG: --manifest-url="" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202302 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202306 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202309 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202313 2570 flags.go:64] FLAG: --max-pods="110" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202316 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202319 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202321 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202324 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202327 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202330 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202333 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202340 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 19:21:28.209174 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202343 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202346 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202349 2570 flags.go:64] FLAG: --pod-cidr="" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202352 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202357 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202360 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202365 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202368 2570 flags.go:64] FLAG: --port="10250" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202371 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202374 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04adfefba3f4d3198" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202377 2570 flags.go:64] FLAG: --qos-reserved="" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202380 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202383 2570 flags.go:64] FLAG: --register-node="true" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202386 2570 flags.go:64] FLAG: --register-with-taints="" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202391 2570 flags.go:64] FLAG: --registry-burst="10" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202394 2570 flags.go:64] FLAG: --registry-qps="5" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202397 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202400 2570 flags.go:64] FLAG: --reserved-memory="" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202403 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202406 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202409 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202412 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202415 2570 flags.go:64] FLAG: --runonce="false" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202418 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202421 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 19:21:28.209796 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202424 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202426 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202429 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202432 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202435 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202438 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202441 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202444 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202447 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202450 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202453 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202456 2570 flags.go:64] FLAG: --system-cgroups="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202459 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202465 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202468 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202471 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202475 2570 flags.go:64] FLAG: --tls-min-version="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202478 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202481 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202484 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202487 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202491 2570 flags.go:64] FLAG: --v="2" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202495 2570 flags.go:64] FLAG: --version="false" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202499 2570 flags.go:64] FLAG: --vmodule="" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202509 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 19:21:28.210412 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.202513 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202659 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202664 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202667 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202670 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202673 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202676 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202680 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202684 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202687 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202690 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202693 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202696 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202700 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202703 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202705 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202708 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202711 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:21:28.211066 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202714 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202716 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202723 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202725 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202728 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202730 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202733 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202736 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202738 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202741 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202745 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202748 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202750 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202754 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202756 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202759 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202761 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202764 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202767 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202769 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:21:28.211511 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202772 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202774 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202777 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202780 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202782 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202785 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202787 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202790 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202792 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202794 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202797 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202800 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202804 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202807 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202811 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202814 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202817 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202819 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202822 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:21:28.212008 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202825 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202827 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202830 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202834 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202836 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202839 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202841 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202844 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202847 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202849 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202852 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202854 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202857 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202859 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202862 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202864 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202867 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202869 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202872 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202874 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:21:28.212466 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202877 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202879 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202882 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202884 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202887 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202890 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202892 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202896 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202898 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202901 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202903 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202906 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202909 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202911 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202914 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202918 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202920 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202924 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202927 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202929 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:21:28.212955 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202932 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202935 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.202937 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.203558 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.210157 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.210173 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.211726 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.211734 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211793 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211798 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211803 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211806 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211809 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211812 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:21:28.213448 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211814 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211817 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211820 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211823 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211826 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211829 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211832 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211834 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211837 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211839 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211842 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211845 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211847 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211850 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211852 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211855 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211857 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211860 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211863 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211866 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:21:28.213826 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211869 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211871 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211874 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211877 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211880 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211882 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211885 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211887 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211890 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211892 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211895 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211897 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211900 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211902 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211905 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211908 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211912 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211915 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211918 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:21:28.214306 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211920 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211923 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211925 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211927 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211930 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211932 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211935 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211938 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211940 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211943 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211945 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211947 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211951 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211955 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211958 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211961 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211964 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211967 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211969 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:21:28.214872 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211972 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211975 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211977 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211980 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211982 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211985 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211988 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211990 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211993 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211996 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.211998 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212001 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212003 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212006 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212009 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212012 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212014 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212017 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212020 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212022 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:21:28.215320 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212025 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212028 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212030 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212033 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212035 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212038 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212041 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212044 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212046 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212049 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212052 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212054 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212058 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212061 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212064 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:21:28.215815 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.212069 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212197 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212201 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212204 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212207 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212210 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212214 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212217 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212220 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212222 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212225 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212227 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212230 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212232 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212235 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212237 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212240 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212242 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212245 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212247 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 19:21:28.216188 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212250 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212253 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212255 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212258 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212261 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212263 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212266 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212269 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212271 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212274 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212276 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212278 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212282 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212286 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212289 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212291 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212294 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212297 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212300 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212302 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 19:21:28.216676 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212305 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212307 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212310 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212312 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212314 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212317 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212319 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212322 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212324 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212327 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212329 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212332 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212335 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212338 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212340 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212343 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212345 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212350 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212353 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 19:21:28.217159 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212356 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212359 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212361 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212363 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212366 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212368 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212371 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212373 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212376 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212378 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212382 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212385 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212388 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212391 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212393 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212396 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212398 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212401 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212404 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212407 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 19:21:28.217641 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212409 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212412 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212414 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212417 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212419 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212422 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212425 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212427 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212430 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212433 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212436 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212439 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212441 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212444 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212446 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212449 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212451 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212454 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212456 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212458 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 19:21:28.218130 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:28.212461 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 19:21:28.218617 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.212466 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 19:21:28.218617 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.212572 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 19:21:28.218617 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.214535 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 19:21:28.218617 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.215999 2570 server.go:1019] "Starting client certificate rotation" Jul 01 19:21:28.218617 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.216098 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 19:21:28.218617 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.216129 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 19:21:28.218910 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.218824 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 19:21:28.220423 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.220404 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 19:21:28.231732 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.231710 2570 log.go:25] "Validated CRI v1 runtime API" Jul 01 19:21:28.235125 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.235108 2570 log.go:25] "Validated CRI v1 image API" Jul 01 19:21:28.235966 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.235952 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 19:21:28.241976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.241954 2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9e0a70a2-d4dc-44a3-883f-861f6daea892:/dev/nvme0n1p4 b39a38c7-6caf-497c-a2d6-e12aaf990c2f:/dev/nvme0n1p3] Jul 01 19:21:28.242046 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.241975 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 19:21:28.246810 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.246794 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 19:21:28.248107 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.248002 2570 manager.go:216] Machine: {Timestamp:2026-07-01 19:21:28.246658193 +0000 UTC m=+0.359206782 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099659 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b1a69821170a5252f965a3fa0c0c4 SystemUUID:ec2b1a69-8211-70a5-252f-965a3fa0c0c4 BootID:a448c1d3-dd11-4c97-97a2-1606111be501 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:12:8f:c0:84:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:12:8f:c0:84:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:54:d4:62:c1:6f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 19:21:28.248107 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.248103 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 19:21:28.248215 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.248181 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 19:21:28.251338 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.251317 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 19:21:28.251478 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.251341 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 19:21:28.252292 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.252282 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 19:21:28.252333 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.252293 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 19:21:28.252333 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.252301 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 19:21:28.253418 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.253408 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 19:21:28.254212 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.254203 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 19:21:28.257662 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257651 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 01 19:21:28.257777 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257768 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 19:21:28.257805 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257798 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 01 19:21:28.257859 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257815 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 19:21:28.257859 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257834 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 19:21:28.257859 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257846 2570 kubelet.go:398] "Adding apiserver pod source" Jul 01 19:21:28.257859 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.257854 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 19:21:28.258208 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.258195 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 19:21:28.258245 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.258214 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 19:21:28.260467 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.260451 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 19:21:28.261940 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.261927 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 19:21:28.261989 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.261957 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 19:21:28.262762 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262752 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262767 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262773 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262779 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262788 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262794 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262803 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262808 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 19:21:28.262811 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262814 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 19:21:28.263019 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262820 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 19:21:28.263019 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262829 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 19:21:28.263019 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.262835 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 19:21:28.263742 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.263731 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 19:21:28.263774 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.263743 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 19:21:28.266896 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.266883 2570 server.go:1268] "Started kubelet" Jul 01 19:21:28.267001 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.266967 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 19:21:28.267001 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.266985 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 19:21:28.272419 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.272394 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 19:21:28.273085 ip-10-0-139-28 systemd[1]: Started Kubernetes Kubelet. Jul 01 19:21:28.273524 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.273486 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 19:21:28.273941 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.273747 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 19:21:28.273941 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.273800 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 19:21:28.273941 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.273806 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xb6sp" Jul 01 19:21:28.277499 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.277477 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 19:21:28.278178 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.278165 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 01 19:21:28.280497 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.280478 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 19:21:28.281519 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.280055 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-28.ec2.internal.18be40be3b393702 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-28.ec2.internal,UID:ip-10-0-139-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-28.ec2.internal,},FirstTimestamp:2026-07-01 19:21:28.266864386 +0000 UTC m=+0.379413024,LastTimestamp:2026-07-01 19:21:28.266864386 +0000 UTC m=+0.379413024,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-28.ec2.internal,}" Jul 01 19:21:28.282128 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.282111 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xb6sp" Jul 01 19:21:28.283311 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.283290 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 19:21:28.283311 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.283309 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 19:21:28.283456 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.283381 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.283983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.283878 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 19:21:28.283983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.283897 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 19:21:28.284526 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284050 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 01 19:21:28.284526 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284058 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 01 19:21:28.284526 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284411 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 19:21:28.284526 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284423 2570 factory.go:55] Registering systemd factory Jul 01 19:21:28.284526 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284444 2570 factory.go:223] Registration of the systemd container factory successfully Jul 01 19:21:28.284847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284564 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 19:21:28.284847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284682 2570 factory.go:153] Registering CRI-O factory Jul 01 19:21:28.284847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284694 2570 factory.go:223] Registration of the crio container factory successfully Jul 01 19:21:28.284847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284712 2570 factory.go:103] Registering Raw factory Jul 01 19:21:28.284847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.284725 2570 manager.go:1196] Started watching for new ooms in manager Jul 01 19:21:28.285167 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.285155 2570 manager.go:349] Starting recovery of all containers Jul 01 19:21:28.286212 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.286188 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 19:21:28.286577 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.286558 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 19:21:28.296682 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.296668 2570 manager.go:354] Recovery completed Jul 01 19:21:28.301778 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.301755 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.303920 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.303899 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.303990 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.303939 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.303990 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.303953 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.304535 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.304520 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 19:21:28.304535 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.304533 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 19:21:28.304673 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.304536 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.304673 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.304548 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 01 19:21:28.305362 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.305344 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.305436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.305371 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.305436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.305380 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.305616 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.305601 2570 policy_none.go:49] "None policy: Start" Jul 01 19:21:28.305694 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.305619 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 19:21:28.305694 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.305644 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 19:21:28.306639 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.306618 2570 policy_none.go:47] "Start" Jul 01 19:21:28.341216 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341190 2570 manager.go:337] "Starting Device Plugin manager" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.341246 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341262 2570 server.go:87] "Starting device plugin registration server" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341506 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341519 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341601 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341692 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.341700 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.342305 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.342357 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.353681 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 19:21:28.354899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.354865 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 19:21:28.355345 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.354912 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 19:21:28.355345 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.354936 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 19:21:28.355345 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.354976 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 19:21:28.357173 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.357157 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:28.442395 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.442363 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.443422 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.443359 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.443422 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.443389 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.443422 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.443402 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.443422 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.443424 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.452465 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.452445 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.452519 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.452469 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-28.ec2.internal\": node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.455154 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.455135 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal"] Jul 01 19:21:28.455202 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.455195 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.456122 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.456105 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.456208 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.456134 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.456208 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.456148 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.457303 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457291 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.457433 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457420 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.457466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457449 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.457954 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457929 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.457954 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457951 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.458047 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457963 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.458047 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457978 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.458047 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.457999 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.458047 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.458008 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.459029 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.459013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.459117 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.459037 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 19:21:28.459669 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.459656 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Jul 01 19:21:28.459741 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.459681 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 19:21:28.459741 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.459696 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Jul 01 19:21:28.471973 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.471957 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.484270 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.484248 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-28.ec2.internal\" not found" node="ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.488747 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.488733 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-28.ec2.internal\" not found" node="ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.573098 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.573060 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.584718 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.584690 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e757ec6aacc35d99d6a9719bfafce01a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-28.ec2.internal\" (UID: \"e757ec6aacc35d99d6a9719bfafce01a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.584838 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.584735 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd323a7b0d15c1fb472752ef1a5a873f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"bd323a7b0d15c1fb472752ef1a5a873f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.584838 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.584755 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd323a7b0d15c1fb472752ef1a5a873f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"bd323a7b0d15c1fb472752ef1a5a873f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.674002 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.673968 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.685455 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.685424 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd323a7b0d15c1fb472752ef1a5a873f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"bd323a7b0d15c1fb472752ef1a5a873f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.685576 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.685460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd323a7b0d15c1fb472752ef1a5a873f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"bd323a7b0d15c1fb472752ef1a5a873f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.685576 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.685483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e757ec6aacc35d99d6a9719bfafce01a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-28.ec2.internal\" (UID: \"e757ec6aacc35d99d6a9719bfafce01a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.685576 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.685523 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd323a7b0d15c1fb472752ef1a5a873f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"bd323a7b0d15c1fb472752ef1a5a873f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.685576 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.685534 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e757ec6aacc35d99d6a9719bfafce01a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-28.ec2.internal\" (UID: \"e757ec6aacc35d99d6a9719bfafce01a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.685576 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.685534 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd323a7b0d15c1fb472752ef1a5a873f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"bd323a7b0d15c1fb472752ef1a5a873f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.775154 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.775074 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.787465 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.787439 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.794885 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:28.794866 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Jul 01 19:21:28.875449 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.875412 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:28.975942 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:28.975905 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:29.076555 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:29.076473 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:29.140347 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.140316 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:29.144095 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.144079 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:29.177010 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:29.176981 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:29.215509 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.215475 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 19:21:29.215958 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.215621 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 19:21:29.215958 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.215680 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 19:21:29.215958 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.215680 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 19:21:29.277615 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:29.277586 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:29.286685 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.286645 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 19:16:28 +0000 UTC" deadline="2028-04-10 20:59:42.640346512 +0000 UTC" Jul 01 19:21:29.286685 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.286681 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15577h38m13.35366864s" Jul 01 19:21:29.378161 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:29.378130 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Jul 01 19:21:29.470373 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:29.470342 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd323a7b0d15c1fb472752ef1a5a873f.slice/crio-71f7f1f2c85ebaacaeb66a4c8a3d14b434b67226ba35c8ed3b64eeefad0d16fa WatchSource:0}: Error finding container 71f7f1f2c85ebaacaeb66a4c8a3d14b434b67226ba35c8ed3b64eeefad0d16fa: Status 404 returned error can't find the container with id 71f7f1f2c85ebaacaeb66a4c8a3d14b434b67226ba35c8ed3b64eeefad0d16fa Jul 01 19:21:29.470599 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:29.470576 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode757ec6aacc35d99d6a9719bfafce01a.slice/crio-f1629e6f1a3d550d0ac28fc5af2009fb0bff1c9753a4dfb8c7a18bf7e67ae202 WatchSource:0}: Error finding container f1629e6f1a3d550d0ac28fc5af2009fb0bff1c9753a4dfb8c7a18bf7e67ae202: Status 404 returned error can't find the container with id f1629e6f1a3d550d0ac28fc5af2009fb0bff1c9753a4dfb8c7a18bf7e67ae202 Jul 01 19:21:29.472404 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.472383 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:29.476766 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.476747 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 19:21:29.483772 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.483754 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Jul 01 19:21:29.499211 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.499187 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 19:21:29.500229 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.500217 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Jul 01 19:21:29.515376 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:29.515356 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 19:21:30.259118 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.259031 2570 apiserver.go:52] "Watching apiserver" Jul 01 19:21:30.273039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.272016 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 19:21:30.273039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.272563 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vnj75","kube-system/konnectivity-agent-fhmrc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w","openshift-image-registry/node-ca-d5cbr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal","openshift-multus/multus-additional-cni-plugins-l8zkw","openshift-network-operator/iptables-alerter-pzrkd","openshift-ovn-kubernetes/ovnkube-node-27vrc","kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mplxs","openshift-dns/node-resolver-xqnc9","openshift-multus/multus-cj5j8","openshift-multus/network-metrics-daemon-d5xdw","openshift-network-diagnostics/network-check-target-j4rjt"] Jul 01 19:21:30.275111 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.274474 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.276532 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.276131 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.277738 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.277466 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.278958 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.278798 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.281143 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.279906 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 19:21:30.281143 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.280293 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rtv6f\"" Jul 01 19:21:30.281143 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.280514 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.281143 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.280756 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:30.281143 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.280858 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:30.282991 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.281837 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.282991 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.282104 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.282991 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.282360 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.282991 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.282397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.284057 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.283367 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.284057 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.284044 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.284494 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.284799 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6ksd6\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.285006 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n52r5\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.285184 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.285321 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.285556 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.285666 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.285918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9wm8s\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.286063 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.286113 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.286258 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.286314 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:30.286353 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.286340 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.288142 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.287447 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 19:21:30.288142 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.287686 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.288142 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.287904 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 19:21:30.289273 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.288679 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.290328 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.290180 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.290328 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.290284 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:30.290840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.290639 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-frpb9\"" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.293087 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.294252 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295003 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295085 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295325 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295559 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295600 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h72vm\" (UniqueName: \"kubernetes.io/projected/7e8570e9-1bef-478c-9ca8-f55188dbc578-kube-api-access-h72vm\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295635 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0998dc6a-abaf-4aaa-ac92-34ea5269afc3-agent-certs\") pod \"konnectivity-agent-fhmrc\" (UID: \"0998dc6a-abaf-4aaa-ac92-34ea5269afc3\") " pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295718 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-systemd-units\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295758 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-systemd\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-var-lib-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295833 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295883 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-socket-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295935 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-sys-fs\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.295995 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-slash\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296068 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-run-netns\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296113 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-ovn\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-cni-bin\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.296281 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296198 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-etc-selinux\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296244 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jcc8\" (UniqueName: \"kubernetes.io/projected/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-kube-api-access-8jcc8\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-system-cni-dir\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296710 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296743 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-etc-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-cni-netd\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7715c025-a35d-4b2b-9ddf-460287a47e52-ovn-node-metrics-cert\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296840 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-registration-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296872 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.296924 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0998dc6a-abaf-4aaa-ac92-34ea5269afc3-konnectivity-ca\") pod \"konnectivity-agent-fhmrc\" (UID: \"0998dc6a-abaf-4aaa-ac92-34ea5269afc3\") " pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297216 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgc82\" (UniqueName: \"kubernetes.io/projected/11338b07-2eb1-432c-be08-260566adfa05-kube-api-access-hgc82\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297270 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-node-log\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-log-socket\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297365 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-env-overrides\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297396 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-device-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297424 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpf4k\" (UniqueName: \"kubernetes.io/projected/622a52e8-4365-440e-8a21-3c13ff32a70e-kube-api-access-rpf4k\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.297466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297458 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/749f1789-dc86-45ff-b45a-e39051e99144-hosts-file\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297535 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-ovnkube-script-lib\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297572 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-serviceca\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297605 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11338b07-2eb1-432c-be08-260566adfa05-host-slash\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297634 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8cs\" (UniqueName: \"kubernetes.io/projected/749f1789-dc86-45ff-b45a-e39051e99144-kube-api-access-vx8cs\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297688 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297735 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlht\" (UniqueName: \"kubernetes.io/projected/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-kube-api-access-jmlht\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297767 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-host\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297802 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/749f1789-dc86-45ff-b45a-e39051e99144-tmp-dir\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297871 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-kubelet\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297944 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-smxx9\"" Jul 01 19:21:30.298330 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.298237 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5cbjg\"" Jul 01 19:21:30.299039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.297945 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-os-release\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.299039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.298917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11338b07-2eb1-432c-be08-260566adfa05-iptables-alerter-script\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.299039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.298954 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-run-ovn-kubernetes\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.299039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.298987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-ovnkube-config\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.299039 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9f67\" (UniqueName: \"kubernetes.io/projected/7715c025-a35d-4b2b-9ddf-460287a47e52-kube-api-access-w9f67\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.299293 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299086 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-cnibin\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.299293 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.299535 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299514 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 19:21:30.299721 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299664 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 19:21:30.299830 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299814 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.299994 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.299975 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 19:21:30.300159 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.300143 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.301711 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.301207 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 19:21:30.302044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.301982 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 19:21:30.303976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.303225 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 19:21:30.303976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.303365 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9szs4\"" Jul 01 19:21:30.303976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.303792 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n8x4c\"" Jul 01 19:21:30.310372 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.310339 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 19:21:30.346733 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.346702 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbrt6" Jul 01 19:21:30.360519 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.360497 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbrt6" Jul 01 19:21:30.361400 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.361307 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" event={"ID":"e757ec6aacc35d99d6a9719bfafce01a","Type":"ContainerStarted","Data":"f1629e6f1a3d550d0ac28fc5af2009fb0bff1c9753a4dfb8c7a18bf7e67ae202"} Jul 01 19:21:30.362856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.362830 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" event={"ID":"bd323a7b0d15c1fb472752ef1a5a873f","Type":"ContainerStarted","Data":"71f7f1f2c85ebaacaeb66a4c8a3d14b434b67226ba35c8ed3b64eeefad0d16fa"} Jul 01 19:21:30.377365 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.377339 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:30.386325 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.386303 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 19:21:30.399278 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399241 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-sys\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.399420 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-cni-bin\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.399420 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-etc-selinux\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.399420 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399366 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jcc8\" (UniqueName: \"kubernetes.io/projected/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-kube-api-access-8jcc8\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.399420 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-run\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399448 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-etc-selinux\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399480 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bwmr\" (UniqueName: \"kubernetes.io/projected/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-kube-api-access-6bwmr\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399500 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-cni-bin\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399516 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4761df2-8e35-4f4a-a6f7-20363c365ca4-kubelet-config\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399545 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4761df2-8e35-4f4a-a6f7-20363c365ca4-dbus\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399571 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysconfig\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.399618 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399600 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-etc-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-cni-netd\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399711 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399722 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-os-release\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-node-log\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-log-socket\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399844 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-node-log\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399856 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-cni-netd\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-env-overrides\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399887 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-etc-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-systemd\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.399932 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-log-socket\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400087 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400059 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-socket-dir-parent\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-k8s-cni-cncf-io\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400183 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-netns\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400216 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-conf-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400245 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-serviceca\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400298 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-cni-multus\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400325 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-os-release\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400344 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-env-overrides\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400354 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11338b07-2eb1-432c-be08-260566adfa05-iptables-alerter-script\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysctl-conf\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400401 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.400442 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400451 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-cnibin\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400483 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4e0861-3883-435e-a7fc-a7e193bb410c-cni-binary-copy\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400448 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-os-release\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.400545 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:21:30.900501083 +0000 UTC m=+3.013049694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:30.400749 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-etc-kubernetes\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400603 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-ovnkube-config\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-cnibin\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400703 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-modprobe-d\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400725 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-cnibin\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400731 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-tuned\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-sys-fs\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400784 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-serviceca\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400808 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-lib-modules\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400832 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-tmp\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400857 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-kubelet\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400872 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-sys-fs\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400910 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-kubernetes\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-slash\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400964 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-run-netns\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400989 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-ovn\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.401510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.400995 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-slash\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401005 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11338b07-2eb1-432c-be08-260566adfa05-iptables-alerter-script\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgc82\" (UniqueName: \"kubernetes.io/projected/11338b07-2eb1-432c-be08-260566adfa05-kube-api-access-hgc82\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401041 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-var-lib-kubelet\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401047 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-run-netns\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-hostroot\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401064 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-ovn\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-multus-certs\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7715c025-a35d-4b2b-9ddf-460287a47e52-ovn-node-metrics-cert\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-registration-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401161 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-ovnkube-config\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0998dc6a-abaf-4aaa-ac92-34ea5269afc3-konnectivity-ca\") pod \"konnectivity-agent-fhmrc\" (UID: \"0998dc6a-abaf-4aaa-ac92-34ea5269afc3\") " pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-device-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401213 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401258 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-registration-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpf4k\" (UniqueName: \"kubernetes.io/projected/622a52e8-4365-440e-8a21-3c13ff32a70e-kube-api-access-rpf4k\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401339 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-device-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.402373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401375 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-ovnkube-script-lib\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401489 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401495 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401531 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11338b07-2eb1-432c-be08-260566adfa05-host-slash\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401559 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8cs\" (UniqueName: \"kubernetes.io/projected/749f1789-dc86-45ff-b45a-e39051e99144-kube-api-access-vx8cs\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401586 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401599 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11338b07-2eb1-432c-be08-260566adfa05-host-slash\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401614 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysctl-d\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-host\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401781 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-cni-bin\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401821 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlht\" (UniqueName: \"kubernetes.io/projected/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-kube-api-access-jmlht\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401846 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-host\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/749f1789-dc86-45ff-b45a-e39051e99144-tmp-dir\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401896 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-system-cni-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-cni-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401948 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-daemon-config\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.403288 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-kubelet\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402086 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-kubelet\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c24wn\" (UniqueName: \"kubernetes.io/projected/6d4e0861-3883-435e-a7fc-a7e193bb410c-kube-api-access-c24wn\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402140 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402140 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-run-ovn-kubernetes\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.401722 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0998dc6a-abaf-4aaa-ac92-34ea5269afc3-konnectivity-ca\") pod \"konnectivity-agent-fhmrc\" (UID: \"0998dc6a-abaf-4aaa-ac92-34ea5269afc3\") " pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402165 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7715c025-a35d-4b2b-9ddf-460287a47e52-ovnkube-script-lib\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9f67\" (UniqueName: \"kubernetes.io/projected/7715c025-a35d-4b2b-9ddf-460287a47e52-kube-api-access-w9f67\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402185 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/749f1789-dc86-45ff-b45a-e39051e99144-tmp-dir\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402195 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-host-run-ovn-kubernetes\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402260 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h72vm\" (UniqueName: \"kubernetes.io/projected/7e8570e9-1bef-478c-9ca8-f55188dbc578-kube-api-access-h72vm\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402283 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0998dc6a-abaf-4aaa-ac92-34ea5269afc3-agent-certs\") pod \"konnectivity-agent-fhmrc\" (UID: \"0998dc6a-abaf-4aaa-ac92-34ea5269afc3\") " pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402309 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/749f1789-dc86-45ff-b45a-e39051e99144-hosts-file\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402333 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-systemd-units\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.404164 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402356 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-systemd\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402386 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-var-lib-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402412 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402421 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/749f1789-dc86-45ff-b45a-e39051e99144-hosts-file\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-socket-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402449 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-host\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-system-cni-dir\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402500 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-systemd-units\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402509 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-var-lib-openvswitch\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402468 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7715c025-a35d-4b2b-9ddf-460287a47e52-run-systemd\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402543 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402567 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e8570e9-1bef-478c-9ca8-f55188dbc578-system-cni-dir\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402654 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-socket-dir\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.405033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.402757 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e8570e9-1bef-478c-9ca8-f55188dbc578-cni-binary-copy\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.406833 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.406808 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7715c025-a35d-4b2b-9ddf-460287a47e52-ovn-node-metrics-cert\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.406977 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.406927 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0998dc6a-abaf-4aaa-ac92-34ea5269afc3-agent-certs\") pod \"konnectivity-agent-fhmrc\" (UID: \"0998dc6a-abaf-4aaa-ac92-34ea5269afc3\") " pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.430986 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.430955 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jcc8\" (UniqueName: \"kubernetes.io/projected/0f3bf7f8-6c2b-4600-a959-65d0bbd34195-kube-api-access-8jcc8\") pod \"node-ca-d5cbr\" (UID: \"0f3bf7f8-6c2b-4600-a959-65d0bbd34195\") " pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.431921 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.431887 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgc82\" (UniqueName: \"kubernetes.io/projected/11338b07-2eb1-432c-be08-260566adfa05-kube-api-access-hgc82\") pod \"iptables-alerter-pzrkd\" (UID: \"11338b07-2eb1-432c-be08-260566adfa05\") " pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.437354 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.436203 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:21:30.437354 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.436233 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:21:30.437354 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.436249 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:30.437354 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.436329 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:30.936307596 +0000 UTC m=+3.048856197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:30.437354 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.436988 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h72vm\" (UniqueName: \"kubernetes.io/projected/7e8570e9-1bef-478c-9ca8-f55188dbc578-kube-api-access-h72vm\") pod \"multus-additional-cni-plugins-l8zkw\" (UID: \"7e8570e9-1bef-478c-9ca8-f55188dbc578\") " pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.437354 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.437307 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9f67\" (UniqueName: \"kubernetes.io/projected/7715c025-a35d-4b2b-9ddf-460287a47e52-kube-api-access-w9f67\") pod \"ovnkube-node-27vrc\" (UID: \"7715c025-a35d-4b2b-9ddf-460287a47e52\") " pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.439835 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.439803 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlht\" (UniqueName: \"kubernetes.io/projected/ab5edea2-a640-4ed1-8f60-35e06c4f5b33-kube-api-access-jmlht\") pod \"aws-ebs-csi-driver-node-zbm2w\" (UID: \"ab5edea2-a640-4ed1-8f60-35e06c4f5b33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.440282 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.440253 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpf4k\" (UniqueName: \"kubernetes.io/projected/622a52e8-4365-440e-8a21-3c13ff32a70e-kube-api-access-rpf4k\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.440955 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.440927 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8cs\" (UniqueName: \"kubernetes.io/projected/749f1789-dc86-45ff-b45a-e39051e99144-kube-api-access-vx8cs\") pod \"node-resolver-xqnc9\" (UID: \"749f1789-dc86-45ff-b45a-e39051e99144\") " pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.503460 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503420 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-sys\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-run\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503501 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bwmr\" (UniqueName: \"kubernetes.io/projected/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-kube-api-access-6bwmr\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4761df2-8e35-4f4a-a6f7-20363c365ca4-kubelet-config\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503549 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-sys\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4761df2-8e35-4f4a-a6f7-20363c365ca4-dbus\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysconfig\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.503664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503650 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-run\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysconfig\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503656 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-os-release\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503722 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-os-release\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503726 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4761df2-8e35-4f4a-a6f7-20363c365ca4-dbus\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-systemd\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503781 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-systemd\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503782 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4761df2-8e35-4f4a-a6f7-20363c365ca4-kubelet-config\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503798 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-socket-dir-parent\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503826 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-k8s-cni-cncf-io\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-netns\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503902 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-netns\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503906 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-socket-dir-parent\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503909 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-k8s-cni-cncf-io\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-conf-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.503984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-cni-multus\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysctl-conf\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504030 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-conf-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504038 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-cnibin\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504064 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4e0861-3883-435e-a7fc-a7e193bb410c-cni-binary-copy\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504073 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-cni-multus\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-etc-kubernetes\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504115 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-modprobe-d\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504131 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-cnibin\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504140 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-tuned\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504164 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysctl-conf\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504173 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-etc-kubernetes\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504281 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-lib-modules\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504355 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-modprobe-d\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-lib-modules\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-tmp\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-kubelet\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504442 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-kubernetes\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-var-lib-kubelet\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504495 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-hostroot\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504520 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-multus-certs\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.504980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504554 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-kubelet\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504567 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysctl-d\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-host\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504602 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-hostroot\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504608 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-kubernetes\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504617 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-cni-bin\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504647 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-system-cni-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504680 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-var-lib-kubelet\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504694 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-cni-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504721 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-daemon-config\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504758 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-run-multus-certs\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504759 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-host\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c24wn\" (UniqueName: \"kubernetes.io/projected/6d4e0861-3883-435e-a7fc-a7e193bb410c-kube-api-access-c24wn\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504782 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4e0861-3883-435e-a7fc-a7e193bb410c-cni-binary-copy\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504722 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-host-var-lib-cni-bin\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504722 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-sysctl-d\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504818 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-system-cni-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.504870 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-cni-dir\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.505840 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.504909 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:30.506854 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.504976 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:31.004954535 +0000 UTC m=+3.117503118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:30.506854 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.505269 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4e0861-3883-435e-a7fc-a7e193bb410c-multus-daemon-config\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.507246 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.507221 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-tmp\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.507624 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.507585 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-etc-tuned\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.525499 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.525411 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c24wn\" (UniqueName: \"kubernetes.io/projected/6d4e0861-3883-435e-a7fc-a7e193bb410c-kube-api-access-c24wn\") pod \"multus-cj5j8\" (UID: \"6d4e0861-3883-435e-a7fc-a7e193bb410c\") " pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.527332 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.527307 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bwmr\" (UniqueName: \"kubernetes.io/projected/0ef997e1-e939-43d0-aaa5-dee6ebbb2aba-kube-api-access-6bwmr\") pod \"tuned-mplxs\" (UID: \"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba\") " pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.596883 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.596843 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pzrkd" Jul 01 19:21:30.608572 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.608547 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xqnc9" Jul 01 19:21:30.617897 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.617871 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" Jul 01 19:21:30.625756 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.625735 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d5cbr" Jul 01 19:21:30.635075 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.635045 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" Jul 01 19:21:30.643030 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.643008 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:30.657665 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.657633 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:30.666804 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.666782 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mplxs" Jul 01 19:21:30.673769 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.673751 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cj5j8" Jul 01 19:21:30.835515 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.835428 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:30.907357 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:30.907323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:30.907543 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.907483 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:30.907592 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:30.907556 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:21:31.907536942 +0000 UTC m=+4.020085520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:31.008325 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:31.008290 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:31.008539 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:31.008343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:31.008539 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.008497 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:31.008686 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.008573 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:32.008552193 +0000 UTC m=+4.121100787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:31.008686 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.008499 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:21:31.008686 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.008642 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:21:31.008686 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.008657 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:31.008889 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.008714 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:32.008695819 +0000 UTC m=+4.121244420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:31.362799 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:31.362754 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:16:30 +0000 UTC" deadline="2028-01-25 00:21:23.255185475 +0000 UTC" Jul 01 19:21:31.362799 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:31.362791 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13732h59m51.892399837s" Jul 01 19:21:31.691395 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.691362 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4e0861_3883_435e_a7fc_a7e193bb410c.slice/crio-3d12c3d7568e93ccfc53b6bf3a646eb886c3a8f1db7c79bfbf24cdbe0308853d WatchSource:0}: Error finding container 3d12c3d7568e93ccfc53b6bf3a646eb886c3a8f1db7c79bfbf24cdbe0308853d: Status 404 returned error can't find the container with id 3d12c3d7568e93ccfc53b6bf3a646eb886c3a8f1db7c79bfbf24cdbe0308853d Jul 01 19:21:31.694995 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.694967 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8570e9_1bef_478c_9ca8_f55188dbc578.slice/crio-c082d8a3e738510fa108d837e71faaa6c60cc205de03a4574df93272a0f2b963 WatchSource:0}: Error finding container c082d8a3e738510fa108d837e71faaa6c60cc205de03a4574df93272a0f2b963: Status 404 returned error can't find the container with id c082d8a3e738510fa108d837e71faaa6c60cc205de03a4574df93272a0f2b963 Jul 01 19:21:31.695860 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.695833 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7715c025_a35d_4b2b_9ddf_460287a47e52.slice/crio-9e5fd1da475e28a947b3f8857360aa31a7ae665e5dad83a75cd18a08b8501173 WatchSource:0}: Error finding container 9e5fd1da475e28a947b3f8857360aa31a7ae665e5dad83a75cd18a08b8501173: Status 404 returned error can't find the container with id 9e5fd1da475e28a947b3f8857360aa31a7ae665e5dad83a75cd18a08b8501173 Jul 01 19:21:31.697849 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.697766 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749f1789_dc86_45ff_b45a_e39051e99144.slice/crio-2b4702ee7571458ffd86f63a7d510db022f2eaced3f2abad70a985ad3b6c8d05 WatchSource:0}: Error finding container 2b4702ee7571458ffd86f63a7d510db022f2eaced3f2abad70a985ad3b6c8d05: Status 404 returned error can't find the container with id 2b4702ee7571458ffd86f63a7d510db022f2eaced3f2abad70a985ad3b6c8d05 Jul 01 19:21:31.703048 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.702822 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0998dc6a_abaf_4aaa_ac92_34ea5269afc3.slice/crio-0d5f42ab2dc7bc5b51a480eb2197a2d732f1d60873d473c4d93bf365d9917b63 WatchSource:0}: Error finding container 0d5f42ab2dc7bc5b51a480eb2197a2d732f1d60873d473c4d93bf365d9917b63: Status 404 returned error can't find the container with id 0d5f42ab2dc7bc5b51a480eb2197a2d732f1d60873d473c4d93bf365d9917b63 Jul 01 19:21:31.703804 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.703772 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5edea2_a640_4ed1_8f60_35e06c4f5b33.slice/crio-3fc3ac778f2fafa2b2002bf15e783aea972813eb9e477d8179d854644b8c6a9f WatchSource:0}: Error finding container 3fc3ac778f2fafa2b2002bf15e783aea972813eb9e477d8179d854644b8c6a9f: Status 404 returned error can't find the container with id 3fc3ac778f2fafa2b2002bf15e783aea972813eb9e477d8179d854644b8c6a9f Jul 01 19:21:31.704570 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.704547 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11338b07_2eb1_432c_be08_260566adfa05.slice/crio-d98851a3b3dbe665ccb805b368e51891bede33c80f10ea24352b1b6e1efdafaa WatchSource:0}: Error finding container d98851a3b3dbe665ccb805b368e51891bede33c80f10ea24352b1b6e1efdafaa: Status 404 returned error can't find the container with id d98851a3b3dbe665ccb805b368e51891bede33c80f10ea24352b1b6e1efdafaa Jul 01 19:21:31.706906 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:21:31.706880 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3bf7f8_6c2b_4600_a959_65d0bbd34195.slice/crio-e806af058aef2051b64c82d75ad75a85c2e894a775bfc4c5704c1c340bbf1c8f WatchSource:0}: Error finding container e806af058aef2051b64c82d75ad75a85c2e894a775bfc4c5704c1c340bbf1c8f: Status 404 returned error can't find the container with id e806af058aef2051b64c82d75ad75a85c2e894a775bfc4c5704c1c340bbf1c8f Jul 01 19:21:31.724599 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:31.724580 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 19:21:31.915669 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:31.915635 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:31.915817 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.915778 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:31.915862 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:31.915850 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:21:33.915833452 +0000 UTC m=+6.028382029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:32.016792 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.016504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:32.016975 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.016819 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:32.016975 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.016694 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:21:32.016975 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.016890 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:21:32.016975 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.016908 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:32.016975 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.016969 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:32.017226 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.016980 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:34.016960148 +0000 UTC m=+6.129508741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:32.017226 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.017020 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:34.017005019 +0000 UTC m=+6.129553613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:32.359815 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.358788 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:32.359815 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.358922 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:32.359815 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.359366 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:32.359815 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.359473 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:32.359815 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.359544 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:32.359815 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:32.359629 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:32.363483 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.363404 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 19:16:30 +0000 UTC" deadline="2027-11-29 17:25:32.814074288 +0000 UTC" Jul 01 19:21:32.363483 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.363454 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12382h4m0.450624403s" Jul 01 19:21:32.384762 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.384724 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pzrkd" event={"ID":"11338b07-2eb1-432c-be08-260566adfa05","Type":"ContainerStarted","Data":"d98851a3b3dbe665ccb805b368e51891bede33c80f10ea24352b1b6e1efdafaa"} Jul 01 19:21:32.397912 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.397820 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fhmrc" event={"ID":"0998dc6a-abaf-4aaa-ac92-34ea5269afc3","Type":"ContainerStarted","Data":"0d5f42ab2dc7bc5b51a480eb2197a2d732f1d60873d473c4d93bf365d9917b63"} Jul 01 19:21:32.403378 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.403230 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerStarted","Data":"c082d8a3e738510fa108d837e71faaa6c60cc205de03a4574df93272a0f2b963"} Jul 01 19:21:32.415229 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.415117 2570 generic.go:356] "Generic (PLEG): container finished" podID="bd323a7b0d15c1fb472752ef1a5a873f" containerID="c08dcf2771305f2cd37e2da05ecba1de93c6b50bf0100887a345ef112ddeb79e" exitCode=0 Jul 01 19:21:32.415229 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.415164 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" event={"ID":"bd323a7b0d15c1fb472752ef1a5a873f","Type":"ContainerDied","Data":"c08dcf2771305f2cd37e2da05ecba1de93c6b50bf0100887a345ef112ddeb79e"} Jul 01 19:21:32.423793 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.423339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d5cbr" event={"ID":"0f3bf7f8-6c2b-4600-a959-65d0bbd34195","Type":"ContainerStarted","Data":"e806af058aef2051b64c82d75ad75a85c2e894a775bfc4c5704c1c340bbf1c8f"} Jul 01 19:21:32.433561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.433457 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" event={"ID":"ab5edea2-a640-4ed1-8f60-35e06c4f5b33","Type":"ContainerStarted","Data":"3fc3ac778f2fafa2b2002bf15e783aea972813eb9e477d8179d854644b8c6a9f"} Jul 01 19:21:32.454628 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.454528 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mplxs" event={"ID":"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba","Type":"ContainerStarted","Data":"9850aa5cef710b920e82d22647316e12088f3ea8957422fb5b53cc508b8e9415"} Jul 01 19:21:32.471096 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.471043 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xqnc9" event={"ID":"749f1789-dc86-45ff-b45a-e39051e99144","Type":"ContainerStarted","Data":"2b4702ee7571458ffd86f63a7d510db022f2eaced3f2abad70a985ad3b6c8d05"} Jul 01 19:21:32.505084 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.504940 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"9e5fd1da475e28a947b3f8857360aa31a7ae665e5dad83a75cd18a08b8501173"} Jul 01 19:21:32.523679 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.523582 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cj5j8" event={"ID":"6d4e0861-3883-435e-a7fc-a7e193bb410c","Type":"ContainerStarted","Data":"3d12c3d7568e93ccfc53b6bf3a646eb886c3a8f1db7c79bfbf24cdbe0308853d"} Jul 01 19:21:32.544659 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.543938 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" event={"ID":"e757ec6aacc35d99d6a9719bfafce01a","Type":"ContainerStarted","Data":"bbf77b112669d69e6c2d8223bbcfd6fcea235d048b1ac1740dd8a3726683e320"} Jul 01 19:21:32.567119 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:32.566913 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" podStartSLOduration=2.566887771 podStartE2EDuration="2.566887771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:32.5660036 +0000 UTC m=+4.678552204" watchObservedRunningTime="2026-07-01 19:21:32.566887771 +0000 UTC m=+4.679436377" Jul 01 19:21:33.587551 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:33.587516 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" event={"ID":"bd323a7b0d15c1fb472752ef1a5a873f","Type":"ContainerStarted","Data":"9a6b18dc42c51726f0dd3d5c4a897e2a713ff6e4af193cad4f7d35d4c7e63255"} Jul 01 19:21:33.936586 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:33.935897 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:33.936586 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:33.936067 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:33.936586 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:33.936138 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:21:37.936116758 +0000 UTC m=+10.048665339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:34.036349 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:34.036308 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:34.036530 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:34.036366 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:34.036530 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.036491 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:34.036652 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.036562 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:38.036539503 +0000 UTC m=+10.149088100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:34.037339 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.037020 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:21:34.037339 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.037043 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:21:34.037339 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.037057 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:34.037339 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.037104 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:38.037088662 +0000 UTC m=+10.149637247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:34.356117 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:34.356031 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:34.356316 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.356179 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:34.356653 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:34.356613 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:34.356817 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.356758 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:34.356906 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:34.356828 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:34.356960 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:34.356902 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:36.359434 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:36.359312 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:36.359434 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:36.359347 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:36.359977 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:36.359317 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:36.359977 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:36.359465 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:36.359977 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:36.359578 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:36.359977 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:36.359676 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:37.970793 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:37.970756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:37.971270 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:37.970883 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:37.971270 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:37.970963 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:21:45.970941982 +0000 UTC m=+18.083490576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:38.071789 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:38.071720 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:38.071983 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.071907 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:38.071983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:38.071933 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:38.072113 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.071992 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:46.071967614 +0000 UTC m=+18.184516198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:38.072113 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.072104 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:21:38.072230 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.072126 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:21:38.072230 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.072141 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:38.072230 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.072203 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:21:46.072185383 +0000 UTC m=+18.184733981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:38.359309 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:38.359222 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:38.359468 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:38.359364 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:38.359468 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.359368 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:38.359468 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.359456 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:38.359682 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:38.359483 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:38.359682 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:38.359542 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:40.358643 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:40.358603 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:40.359113 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:40.358747 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:40.359113 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:40.358823 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:40.359113 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:40.358861 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:40.359113 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:40.358964 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:40.359113 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:40.359058 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:42.356227 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:42.356187 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:42.356700 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:42.356347 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:42.356700 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:42.356425 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:42.356700 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:42.356545 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:42.356700 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:42.356576 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:42.356700 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:42.356678 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:43.618785 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.618749 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e8570e9-1bef-478c-9ca8-f55188dbc578" containerID="f5d3089a90e741027578b9db6ec82b80e2074842b4bbc519c906a4e0115fbb3e" exitCode=0 Jul 01 19:21:43.619316 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.618845 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerDied","Data":"f5d3089a90e741027578b9db6ec82b80e2074842b4bbc519c906a4e0115fbb3e"} Jul 01 19:21:43.620497 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.620461 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d5cbr" event={"ID":"0f3bf7f8-6c2b-4600-a959-65d0bbd34195","Type":"ContainerStarted","Data":"eec98510a9ac2c244970e4f923143ceb5247cd11214cee03bee8faf1b1dfda6d"} Jul 01 19:21:43.622329 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.622301 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" event={"ID":"ab5edea2-a640-4ed1-8f60-35e06c4f5b33","Type":"ContainerStarted","Data":"ada8b0330e4c38998d5eec6847ee5083bd6ab80e56d4e1da11a293fc3956b659"} Jul 01 19:21:43.624285 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.624256 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mplxs" event={"ID":"0ef997e1-e939-43d0-aaa5-dee6ebbb2aba","Type":"ContainerStarted","Data":"ab11546405c63fc892c144938d6e7434119578e65ab38e7b69cf337950242d3f"} Jul 01 19:21:43.626684 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.626547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xqnc9" event={"ID":"749f1789-dc86-45ff-b45a-e39051e99144","Type":"ContainerStarted","Data":"fedb0382a42c411da5fd1b5ee28c0094ae3abebb59b1df19af29ddf99de22240"} Jul 01 19:21:43.629051 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.629029 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fhmrc" event={"ID":"0998dc6a-abaf-4aaa-ac92-34ea5269afc3","Type":"ContainerStarted","Data":"59e6755c9e74420310eac4185c99aae59d7065796f70e4e87bb55aae008be89e"} Jul 01 19:21:43.648954 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.647510 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" podStartSLOduration=13.647493724 podStartE2EDuration="13.647493724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:33.621452819 +0000 UTC m=+5.734001425" watchObservedRunningTime="2026-07-01 19:21:43.647493724 +0000 UTC m=+15.760042323" Jul 01 19:21:43.668584 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.668537 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xqnc9" podStartSLOduration=3.85435302 podStartE2EDuration="14.668521213s" totalImagesPullingTime="10.814168193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:43.668082168 +0000 UTC m=+15.780630767" watchObservedRunningTime="2026-07-01 19:21:43.668521213 +0000 UTC m=+15.781069812" Jul 01 19:21:43.692418 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.692362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d5cbr" podStartSLOduration=3.887887032 podStartE2EDuration="14.692343135s" totalImagesPullingTime="10.804456103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:43.692006281 +0000 UTC m=+15.804554889" watchObservedRunningTime="2026-07-01 19:21:43.692343135 +0000 UTC m=+15.804891734" Jul 01 19:21:43.752134 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:43.751800 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mplxs" podStartSLOduration=3.924243904 podStartE2EDuration="14.751778381s" totalImagesPullingTime="10.827534477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:43.71942618 +0000 UTC m=+15.831974781" watchObservedRunningTime="2026-07-01 19:21:43.751778381 +0000 UTC m=+15.864326981" Jul 01 19:21:44.359036 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:44.359008 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:44.359197 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:44.359072 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:44.359197 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:44.359079 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:44.359197 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:44.359172 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:44.359458 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:44.359422 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:44.359582 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:44.359525 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:44.633691 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:44.633595 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pzrkd" event={"ID":"11338b07-2eb1-432c-be08-260566adfa05","Type":"ContainerStarted","Data":"10eeda38aa08bb6fe7f4dce09142abd6d72c9dec42c6f8ee4635eb29d4d35cf0"} Jul 01 19:21:44.652722 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:44.652672 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fhmrc" podStartSLOduration=5.176129131 podStartE2EDuration="15.652653614s" totalImagesPullingTime="10.476524483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:43.752377699 +0000 UTC m=+15.864926290" watchObservedRunningTime="2026-07-01 19:21:44.652653614 +0000 UTC m=+16.765202214" Jul 01 19:21:44.653029 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:44.653001 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pzrkd" podStartSLOduration=4.84697228 podStartE2EDuration="15.65299189s" totalImagesPullingTime="10.80601961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:44.652992231 +0000 UTC m=+16.765540854" watchObservedRunningTime="2026-07-01 19:21:44.65299189 +0000 UTC m=+16.765540489" Jul 01 19:21:45.898803 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:45.898767 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:45.899483 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:45.899458 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:46.033598 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.033562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:46.033966 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.033889 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:46.034117 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.033991 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:02.033958361 +0000 UTC m=+34.146506939 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:21:46.134223 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.134183 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:46.134433 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.134239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:46.134433 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.134354 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:46.134433 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.134376 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:21:46.134433 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.134400 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:21:46.134433 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.134416 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:46.134433 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.134420 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:02.134401161 +0000 UTC m=+34.246949753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:21:46.134681 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.134459 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:02.134445764 +0000 UTC m=+34.246994342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:21:46.356066 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.355982 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:46.356066 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.356012 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:46.356280 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.355994 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:46.356280 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.356121 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:46.356280 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.356240 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:46.356417 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:46.356319 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:46.638078 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.637992 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:46.638535 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:46.638512 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fhmrc" Jul 01 19:21:48.196780 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:48.196724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fhmrc_0998dc6a-abaf-4aaa-ac92-34ea5269afc3/konnectivity-agent/0.log" limit=1024 Jul 01 19:21:48.293827 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:48.293723 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fhmrc_0998dc6a-abaf-4aaa-ac92-34ea5269afc3/konnectivity-agent/0.log" limit=1024 Jul 01 19:21:48.356346 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:48.356310 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:48.356522 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:48.356401 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:48.356522 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:48.356467 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:48.356522 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:48.356476 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:48.356522 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:48.356501 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:48.356774 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:48.356566 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:49.239639 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:49.239594 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fhmrc_0998dc6a-abaf-4aaa-ac92-34ea5269afc3/konnectivity-agent/0.log" limit=1024 Jul 01 19:21:50.355670 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:50.355639 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:50.356138 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:50.355639 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:50.356138 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:50.355779 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:50.356138 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:50.355649 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:50.356138 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:50.355865 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:50.356138 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:50.356005 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:52.356015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:52.355973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:52.356587 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:52.355985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:52.356587 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:52.356114 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:52.356587 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:52.355985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:52.356587 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:52.356198 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:52.356587 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:52.356259 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:54.355821 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.355792 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:54.356232 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.355830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:54.356232 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.355792 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:54.356232 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:54.355902 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:54.356232 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:54.356006 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:54.356232 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:54.356063 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:54.498036 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.497816 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 19:21:54.658752 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.658369 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" event={"ID":"ab5edea2-a640-4ed1-8f60-35e06c4f5b33","Type":"ContainerStarted","Data":"c47bdebe42ea3e2f0aa331a3b1feafda4190289a7c09f6d8fcf976764a4717cd"} Jul 01 19:21:54.661566 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.661540 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:21:54.661945 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.661920 2570 generic.go:356] "Generic (PLEG): container finished" podID="7715c025-a35d-4b2b-9ddf-460287a47e52" containerID="0fcf939f38bbdac03119c772b1af3993fea19f171f8a383b4b89982771251d5d" exitCode=1 Jul 01 19:21:54.662015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.661979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"532cf078ba8e76799efd38a2c9160e54711dc754184df884dffaf4dbed9246d7"} Jul 01 19:21:54.662086 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.662073 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerDied","Data":"0fcf939f38bbdac03119c772b1af3993fea19f171f8a383b4b89982771251d5d"} Jul 01 19:21:54.662148 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.662131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"d317db93be488838fd652c2943e61209ff74b25935e0d2156da4597e84e72bcd"} Jul 01 19:21:54.663590 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.663564 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cj5j8" event={"ID":"6d4e0861-3883-435e-a7fc-a7e193bb410c","Type":"ContainerStarted","Data":"422cebaabbd0d698fc55a5386f499206b41612743dc9b29f1faa8398a2a2c718"} Jul 01 19:21:54.665519 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.665495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerStarted","Data":"6483812236a1d5d2e004c54bdbd6462eb40bbab02377ad2cb64e6921fc2038ad"} Jul 01 19:21:54.683038 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:54.682993 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cj5j8" podStartSLOduration=3.005942714 podStartE2EDuration="25.682979893s" totalImagesPullingTime="22.677037179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:54.681930831 +0000 UTC m=+26.794479431" watchObservedRunningTime="2026-07-01 19:21:54.682979893 +0000 UTC m=+26.795528528" Jul 01 19:21:55.356878 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.356551 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T19:21:54.498022734Z","UUID":"fe42cd3b-c136-4b63-aa66-be2880b95c3f","Handler":null,"Name":"","Endpoint":""} Jul 01 19:21:55.357999 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.357979 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 19:21:55.358104 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.358016 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 19:21:55.670601 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.670573 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:21:55.671088 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.671065 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"bfbcc058d7cdec6b11cb9d4595e635b75f4dfe59d4c18f0ddfa56810cbc898f4"} Jul 01 19:21:55.671236 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.671220 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"6b7ed92232d3e160cdb6a3cb14bfccc95ac60d8bb8297982d7b3912bc1efe2a8"} Jul 01 19:21:55.671306 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.671276 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"a5736ef1777d8a0d8b587dedb5f1ba4eb683f88db9dbc0a79c2bf4ed7ba31c06"} Jul 01 19:21:55.673130 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.673108 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e8570e9-1bef-478c-9ca8-f55188dbc578" containerID="6483812236a1d5d2e004c54bdbd6462eb40bbab02377ad2cb64e6921fc2038ad" exitCode=0 Jul 01 19:21:55.673227 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:55.673203 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerDied","Data":"6483812236a1d5d2e004c54bdbd6462eb40bbab02377ad2cb64e6921fc2038ad"} Jul 01 19:21:56.356134 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:56.356058 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:56.356322 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:56.356058 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:56.356322 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:56.356217 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:56.356322 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:56.356235 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:56.356322 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:56.356062 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:56.356538 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:56.356378 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:56.676528 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:56.676444 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" event={"ID":"ab5edea2-a640-4ed1-8f60-35e06c4f5b33","Type":"ContainerStarted","Data":"f9c6b902570ec30c1b4f6bc093d28fae8789e117f36059df21f9b05e821ca7fc"} Jul 01 19:21:57.682032 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:57.682006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:21:57.682430 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:57.682353 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"cf9ebffbab08690ae83bbc9b1dd1250103487637c411c52cda1da1d2cb2a03b7"} Jul 01 19:21:57.684215 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:57.684193 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e8570e9-1bef-478c-9ca8-f55188dbc578" containerID="27b0a8e5edb6d77c501be35b8b289d125211acb897f5553b0e08449a60f49978" exitCode=0 Jul 01 19:21:57.684361 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:57.684271 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerDied","Data":"27b0a8e5edb6d77c501be35b8b289d125211acb897f5553b0e08449a60f49978"} Jul 01 19:21:57.711334 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:57.711290 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbm2w" podStartSLOduration=5.178510148 podStartE2EDuration="28.711275083s" totalImagesPullingTime="23.532764935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:21:56.699882553 +0000 UTC m=+28.812431153" watchObservedRunningTime="2026-07-01 19:21:57.711275083 +0000 UTC m=+29.823823681" Jul 01 19:21:58.356803 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:58.356772 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:21:58.356983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:58.356818 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:21:58.356983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:58.356925 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:21:58.356983 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:58.356936 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:21:58.357140 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:58.357002 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:21:58.357140 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:21:58.357084 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:21:59.692684 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.692468 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:21:59.693368 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.693037 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"2dfff8f0b8c3a6f3c056fbc8af086857ba7d41ca88d1ac15d48cc54338d4fd93"} Jul 01 19:21:59.693368 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.693285 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:59.693368 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.693352 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:59.693508 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.693398 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:59.693508 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.693465 2570 scope.go:117] "RemoveContainer" containerID="0fcf939f38bbdac03119c772b1af3993fea19f171f8a383b4b89982771251d5d" Jul 01 19:21:59.696112 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.696092 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e8570e9-1bef-478c-9ca8-f55188dbc578" containerID="f9bd43baa88eae328d2b90f9dc05d980e6cdd3864d2e36c666abc3f832b93a2a" exitCode=0 Jul 01 19:21:59.696201 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.696144 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerDied","Data":"f9bd43baa88eae328d2b90f9dc05d980e6cdd3864d2e36c666abc3f832b93a2a"} Jul 01 19:21:59.710983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.710963 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:21:59.715046 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:21:59.715031 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:22:00.355702 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:00.355669 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:00.355894 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:00.355672 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:00.355894 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:00.355785 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:22:00.355894 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:00.355864 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:22:00.355894 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:00.355672 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:00.356053 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:00.356001 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:22:00.703358 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:00.703275 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:22:00.703827 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:00.703684 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" event={"ID":"7715c025-a35d-4b2b-9ddf-460287a47e52","Type":"ContainerStarted","Data":"8b25ae2820c3a555c778ee1b4c59edcf9f3aa6a4007f2b42dcce5ca2368a9901"} Jul 01 19:22:00.744721 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:00.744665 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" podStartSLOduration=9.072568454 podStartE2EDuration="31.744648949s" totalImagesPullingTime="22.672080495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:00.743090498 +0000 UTC m=+32.855639098" watchObservedRunningTime="2026-07-01 19:22:00.744648949 +0000 UTC m=+32.857197554" Jul 01 19:22:01.624919 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:01.624880 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j4rjt"] Jul 01 19:22:01.625072 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:01.624991 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:01.625133 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:01.625084 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:22:01.625599 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:01.625578 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d5xdw"] Jul 01 19:22:01.625686 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:01.625670 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:01.625787 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:01.625766 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:22:01.626097 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:01.626073 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vnj75"] Jul 01 19:22:01.626161 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:01.626152 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:01.626266 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:01.626238 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:22:02.050527 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:02.050488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:02.051010 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.050643 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:22:02.051010 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.050714 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs podName:622a52e8-4365-440e-8a21-3c13ff32a70e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:34.050694785 +0000 UTC m=+66.163243364 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs") pod "network-metrics-daemon-d5xdw" (UID: "622a52e8-4365-440e-8a21-3c13ff32a70e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 19:22:02.151845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:02.151808 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:02.151998 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:02.151911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:02.151998 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.151955 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 19:22:02.152090 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.152029 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret podName:a4761df2-8e35-4f4a-a6f7-20363c365ca4 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:34.152014587 +0000 UTC m=+66.264563166 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret") pod "global-pull-secret-syncer-vnj75" (UID: "a4761df2-8e35-4f4a-a6f7-20363c365ca4") : object "kube-system"/"original-pull-secret" not registered Jul 01 19:22:02.152090 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.152029 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 19:22:02.152090 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.152054 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 19:22:02.152090 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.152064 2570 projected.go:196] Error preparing data for projected volume kube-api-access-8t4r8 for pod openshift-network-diagnostics/network-check-target-j4rjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:22:02.152226 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:02.152104 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8 podName:be8a31ef-4a82-4e5b-9f13-934033eb47a3 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:34.152094091 +0000 UTC m=+66.264642670 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8t4r8" (UniqueName: "kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8") pod "network-check-target-j4rjt" (UID: "be8a31ef-4a82-4e5b-9f13-934033eb47a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 19:22:03.356136 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:03.356102 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:03.356136 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:03.356134 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:03.356590 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:03.356226 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:22:03.356590 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:03.356241 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:03.356590 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:03.356379 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:22:03.356590 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:03.356481 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:22:04.715426 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:04.715173 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e8570e9-1bef-478c-9ca8-f55188dbc578" containerID="8a81f1ab7b1ac487bb55cc36b19da55101d9dacc8f19ccafa40f09b11e0b69c2" exitCode=0 Jul 01 19:22:04.715821 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:04.715258 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerDied","Data":"8a81f1ab7b1ac487bb55cc36b19da55101d9dacc8f19ccafa40f09b11e0b69c2"} Jul 01 19:22:05.355893 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:05.355849 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:05.356065 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:05.355857 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:05.356065 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:05.355967 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:05.356065 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:05.355962 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j4rjt" podUID="be8a31ef-4a82-4e5b-9f13-934033eb47a3" Jul 01 19:22:05.356234 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:05.356063 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5xdw" podUID="622a52e8-4365-440e-8a21-3c13ff32a70e" Jul 01 19:22:05.356234 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:05.356126 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vnj75" podUID="a4761df2-8e35-4f4a-a6f7-20363c365ca4" Jul 01 19:22:05.721153 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:05.721069 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e8570e9-1bef-478c-9ca8-f55188dbc578" containerID="0e030bcbf5393d685ab71bc000e88df07e2faaef8e47151b641298c8b9ddbd8a" exitCode=0 Jul 01 19:22:05.721153 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:05.721123 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerDied","Data":"0e030bcbf5393d685ab71bc000e88df07e2faaef8e47151b641298c8b9ddbd8a"} Jul 01 19:22:06.728122 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:06.728089 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" event={"ID":"7e8570e9-1bef-478c-9ca8-f55188dbc578","Type":"ContainerStarted","Data":"aa07e594bc35c162d45b1609898c9142c422a18f63f56111f84012dd58cdab84"} Jul 01 19:22:06.752967 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:06.752911 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l8zkw" podStartSLOduration=9.676876124 podStartE2EDuration="37.752895769s" totalImagesPullingTime="28.076019645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:06.752707787 +0000 UTC m=+38.865256388" watchObservedRunningTime="2026-07-01 19:22:06.752895769 +0000 UTC m=+38.865444368" Jul 01 19:22:07.153530 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.153502 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeReady" Jul 01 19:22:07.153739 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.153696 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 19:22:07.244967 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.244938 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbh8p"] Jul 01 19:22:07.266119 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.266086 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g9r8t"] Jul 01 19:22:07.266404 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.266221 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.269882 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.269840 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 19:22:07.269882 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.269860 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-d8sfh\"" Jul 01 19:22:07.270674 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.270651 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 19:22:07.270988 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.270974 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 19:22:07.283202 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.283183 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbh8p"] Jul 01 19:22:07.283297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.283211 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.283297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.283230 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g9r8t"] Jul 01 19:22:07.286548 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.286532 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 19:22:07.286548 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.286544 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f8prn\"" Jul 01 19:22:07.286689 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.286532 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 19:22:07.356104 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.356068 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:07.356292 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.356072 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:07.356355 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.356072 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:07.359747 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.359722 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 19:22:07.359747 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.359746 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zbvbl\"" Jul 01 19:22:07.359897 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.359754 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 19:22:07.360095 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.360079 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 19:22:07.360095 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.360086 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 19:22:07.360095 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.360096 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mzk4c\"" Jul 01 19:22:07.388989 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.388952 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.388989 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.388989 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hnbd\" (UniqueName: \"kubernetes.io/projected/d415df21-c25d-439a-8147-9d82e9c13079-kube-api-access-9hnbd\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.389191 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.389019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d415df21-c25d-439a-8147-9d82e9c13079-tmp-dir\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.389191 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.389050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d415df21-c25d-439a-8147-9d82e9c13079-config-volume\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.389191 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.389066 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/909f7802-3f30-4305-888f-7d688cd0e003-cert\") pod \"ingress-canary-gbh8p\" (UID: \"909f7802-3f30-4305-888f-7d688cd0e003\") " pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.389191 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.389081 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nnsd\" (UniqueName: \"kubernetes.io/projected/909f7802-3f30-4305-888f-7d688cd0e003-kube-api-access-2nnsd\") pod \"ingress-canary-gbh8p\" (UID: \"909f7802-3f30-4305-888f-7d688cd0e003\") " pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.489914 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.489828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/909f7802-3f30-4305-888f-7d688cd0e003-cert\") pod \"ingress-canary-gbh8p\" (UID: \"909f7802-3f30-4305-888f-7d688cd0e003\") " pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.489914 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.489861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nnsd\" (UniqueName: \"kubernetes.io/projected/909f7802-3f30-4305-888f-7d688cd0e003-kube-api-access-2nnsd\") pod \"ingress-canary-gbh8p\" (UID: \"909f7802-3f30-4305-888f-7d688cd0e003\") " pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.490135 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.489936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.490135 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.489962 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hnbd\" (UniqueName: \"kubernetes.io/projected/d415df21-c25d-439a-8147-9d82e9c13079-kube-api-access-9hnbd\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.490135 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.489992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d415df21-c25d-439a-8147-9d82e9c13079-tmp-dir\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.490135 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.490037 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d415df21-c25d-439a-8147-9d82e9c13079-config-volume\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.490135 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:07.490076 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:22:07.490352 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:07.490154 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls podName:d415df21-c25d-439a-8147-9d82e9c13079 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:07.990132204 +0000 UTC m=+40.102680784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls") pod "dns-default-g9r8t" (UID: "d415df21-c25d-439a-8147-9d82e9c13079") : secret "dns-default-metrics-tls" not found Jul 01 19:22:07.490418 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.490404 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d415df21-c25d-439a-8147-9d82e9c13079-tmp-dir\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.490600 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.490580 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d415df21-c25d-439a-8147-9d82e9c13079-config-volume\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.494162 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.494142 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/909f7802-3f30-4305-888f-7d688cd0e003-cert\") pod \"ingress-canary-gbh8p\" (UID: \"909f7802-3f30-4305-888f-7d688cd0e003\") " pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.498483 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.498459 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nnsd\" (UniqueName: \"kubernetes.io/projected/909f7802-3f30-4305-888f-7d688cd0e003-kube-api-access-2nnsd\") pod \"ingress-canary-gbh8p\" (UID: \"909f7802-3f30-4305-888f-7d688cd0e003\") " pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.499081 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.499062 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hnbd\" (UniqueName: \"kubernetes.io/projected/d415df21-c25d-439a-8147-9d82e9c13079-kube-api-access-9hnbd\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.577259 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.577222 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbh8p" Jul 01 19:22:07.768479 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.768271 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbh8p"] Jul 01 19:22:07.772577 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:07.772546 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909f7802_3f30_4305_888f_7d688cd0e003.slice/crio-65ce29cfc916b5657074b732079980cfa0a6dfa076b20e1067b75947ff842a4c WatchSource:0}: Error finding container 65ce29cfc916b5657074b732079980cfa0a6dfa076b20e1067b75947ff842a4c: Status 404 returned error can't find the container with id 65ce29cfc916b5657074b732079980cfa0a6dfa076b20e1067b75947ff842a4c Jul 01 19:22:07.827372 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.827323 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2"] Jul 01 19:22:07.836828 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.836803 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" Jul 01 19:22:07.839748 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.839723 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lpxbf\"" Jul 01 19:22:07.839880 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.839845 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 01 19:22:07.839994 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.839979 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 01 19:22:07.847468 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.844533 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2"] Jul 01 19:22:07.993294 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.993260 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:07.993474 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:07.993394 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:22:07.993474 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:07.993418 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6pq4\" (UniqueName: \"kubernetes.io/projected/d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f-kube-api-access-x6pq4\") pod \"migrator-786f9fbc5-nrds2\" (UID: \"d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" Jul 01 19:22:07.993474 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:07.993452 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls podName:d415df21-c25d-439a-8147-9d82e9c13079 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:08.993436498 +0000 UTC m=+41.105985075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls") pod "dns-default-g9r8t" (UID: "d415df21-c25d-439a-8147-9d82e9c13079") : secret "dns-default-metrics-tls" not found Jul 01 19:22:08.094718 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.094593 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6pq4\" (UniqueName: \"kubernetes.io/projected/d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f-kube-api-access-x6pq4\") pod \"migrator-786f9fbc5-nrds2\" (UID: \"d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" Jul 01 19:22:08.103664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.103636 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6pq4\" (UniqueName: \"kubernetes.io/projected/d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f-kube-api-access-x6pq4\") pod \"migrator-786f9fbc5-nrds2\" (UID: \"d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" Jul 01 19:22:08.149530 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.149494 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" Jul 01 19:22:08.303904 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.303872 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2"] Jul 01 19:22:08.307313 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:08.307285 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67cc2f7_1edb_4cb6_a20f_3139c2a71c1f.slice/crio-1a82ae12085e49f6521183be34d70089e4e481951b7c7701f6b57e550d576be5 WatchSource:0}: Error finding container 1a82ae12085e49f6521183be34d70089e4e481951b7c7701f6b57e550d576be5: Status 404 returned error can't find the container with id 1a82ae12085e49f6521183be34d70089e4e481951b7c7701f6b57e550d576be5 Jul 01 19:22:08.447655 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.447563 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zzdzg"] Jul 01 19:22:08.481805 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.481773 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zzdzg"] Jul 01 19:22:08.482141 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.481923 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.485132 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.485106 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 19:22:08.485370 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.485279 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 19:22:08.485370 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.485344 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 19:22:08.485647 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.485609 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 19:22:08.485746 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.485647 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tz26f\"" Jul 01 19:22:08.598725 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.598694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ebd2222-c969-4a62-a631-4a059587a11e-crio-socket\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.598926 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.598752 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfnlc\" (UniqueName: \"kubernetes.io/projected/8ebd2222-c969-4a62-a631-4a059587a11e-kube-api-access-rfnlc\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.598926 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.598884 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.599042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.598941 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.599042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.598974 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ebd2222-c969-4a62-a631-4a059587a11e-data-volume\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699449 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.699369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ebd2222-c969-4a62-a631-4a059587a11e-crio-socket\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699449 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.699410 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfnlc\" (UniqueName: \"kubernetes.io/projected/8ebd2222-c969-4a62-a631-4a059587a11e-kube-api-access-rfnlc\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.699556 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.699640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.699644 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ebd2222-c969-4a62-a631-4a059587a11e-crio-socket\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.699673 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ebd2222-c969-4a62-a631-4a059587a11e-data-volume\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:08.699700 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:08.699738 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:22:08.699772 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:08.699759 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:09.199738352 +0000 UTC m=+41.312286945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : secret "insights-runtime-extractor-tls" not found Jul 01 19:22:08.700080 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:08.699798 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:09.199781603 +0000 UTC m=+41.312330202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : configmap "kube-rbac-proxy" not found Jul 01 19:22:08.700080 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.700005 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ebd2222-c969-4a62-a631-4a059587a11e-data-volume\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.708723 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.708687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfnlc\" (UniqueName: \"kubernetes.io/projected/8ebd2222-c969-4a62-a631-4a059587a11e-kube-api-access-rfnlc\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:08.735303 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.735267 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" event={"ID":"d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f","Type":"ContainerStarted","Data":"1a82ae12085e49f6521183be34d70089e4e481951b7c7701f6b57e550d576be5"} Jul 01 19:22:08.736469 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:08.736439 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbh8p" event={"ID":"909f7802-3f30-4305-888f-7d688cd0e003","Type":"ContainerStarted","Data":"65ce29cfc916b5657074b732079980cfa0a6dfa076b20e1067b75947ff842a4c"} Jul 01 19:22:09.002388 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:09.002304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:09.002831 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:09.002526 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:22:09.002831 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:09.002589 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls podName:d415df21-c25d-439a-8147-9d82e9c13079 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:11.002571372 +0000 UTC m=+43.115119964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls") pod "dns-default-g9r8t" (UID: "d415df21-c25d-439a-8147-9d82e9c13079") : secret "dns-default-metrics-tls" not found Jul 01 19:22:09.203984 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:09.203947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:09.204192 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:09.204006 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:09.204192 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:09.204134 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:22:09.204305 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:09.204195 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:10.204177682 +0000 UTC m=+42.316726273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : configmap "kube-rbac-proxy" not found Jul 01 19:22:09.204537 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:09.204512 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:22:09.204658 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:09.204579 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:10.204565154 +0000 UTC m=+42.317113743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : secret "insights-runtime-extractor-tls" not found Jul 01 19:22:10.103360 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.103326 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-klwbj"] Jul 01 19:22:10.116608 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.116577 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.119428 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.119374 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-klwbj"] Jul 01 19:22:10.119784 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.119753 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 01 19:22:10.119784 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.119753 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 01 19:22:10.119955 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.119819 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 01 19:22:10.120906 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.120884 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4zs26\"" Jul 01 19:22:10.121010 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.120912 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 01 19:22:10.212958 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.212908 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:10.213146 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.212966 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/503c7647-a1dd-45c8-8574-c5e09b8426ac-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.213146 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.212999 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppjr\" (UniqueName: \"kubernetes.io/projected/503c7647-a1dd-45c8-8574-c5e09b8426ac-kube-api-access-cppjr\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.213146 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.213096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:10.213310 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:10.213107 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:22:10.213310 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.213158 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/503c7647-a1dd-45c8-8574-c5e09b8426ac-signing-key\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.213310 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:10.213146 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:22:10.213310 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:10.213243 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:12.213215738 +0000 UTC m=+44.325764315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : secret "insights-runtime-extractor-tls" not found Jul 01 19:22:10.213502 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:10.213331 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:12.213317807 +0000 UTC m=+44.325866383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : configmap "kube-rbac-proxy" not found Jul 01 19:22:10.313718 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.313685 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/503c7647-a1dd-45c8-8574-c5e09b8426ac-signing-key\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.313887 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.313785 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/503c7647-a1dd-45c8-8574-c5e09b8426ac-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.313887 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.313807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppjr\" (UniqueName: \"kubernetes.io/projected/503c7647-a1dd-45c8-8574-c5e09b8426ac-kube-api-access-cppjr\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.314472 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.314440 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/503c7647-a1dd-45c8-8574-c5e09b8426ac-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.316471 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.316452 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/503c7647-a1dd-45c8-8574-c5e09b8426ac-signing-key\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.322865 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.322845 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppjr\" (UniqueName: \"kubernetes.io/projected/503c7647-a1dd-45c8-8574-c5e09b8426ac-kube-api-access-cppjr\") pod \"service-ca-7bb4f4bcc7-klwbj\" (UID: \"503c7647-a1dd-45c8-8574-c5e09b8426ac\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.428415 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.428330 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" Jul 01 19:22:10.823148 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:10.823082 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-klwbj"] Jul 01 19:22:10.827609 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:10.827576 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503c7647_a1dd_45c8_8574_c5e09b8426ac.slice/crio-9536f0c286e2c59c01c379ba50cbd1648335710de97cab3351bd58120fc4fb87 WatchSource:0}: Error finding container 9536f0c286e2c59c01c379ba50cbd1648335710de97cab3351bd58120fc4fb87: Status 404 returned error can't find the container with id 9536f0c286e2c59c01c379ba50cbd1648335710de97cab3351bd58120fc4fb87 Jul 01 19:22:11.018893 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.018864 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:11.019053 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:11.019002 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:22:11.019109 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:11.019063 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls podName:d415df21-c25d-439a-8147-9d82e9c13079 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:15.019047497 +0000 UTC m=+47.131596074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls") pod "dns-default-g9r8t" (UID: "d415df21-c25d-439a-8147-9d82e9c13079") : secret "dns-default-metrics-tls" not found Jul 01 19:22:11.416870 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.416834 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xqnc9_749f1789-dc86-45ff-b45a-e39051e99144/dns-node-resolver/0.log" Jul 01 19:22:11.746203 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.746125 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" event={"ID":"d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f","Type":"ContainerStarted","Data":"af6bdb29c460e07b17e3af10862d430b58918f05ba4dac1b3166c16edd7876c2"} Jul 01 19:22:11.746203 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.746193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" event={"ID":"d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f","Type":"ContainerStarted","Data":"319868c1043c2f5ab8f5caa169f7857a10e410475968f3e50da590646c6f6bc6"} Jul 01 19:22:11.747810 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.747775 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbh8p" event={"ID":"909f7802-3f30-4305-888f-7d688cd0e003","Type":"ContainerStarted","Data":"ea0f7a85fc550770a29b5f3fc66dfb865b7b797496ca524f5e910c3947a2a1a5"} Jul 01 19:22:11.748919 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.748889 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" event={"ID":"503c7647-a1dd-45c8-8574-c5e09b8426ac","Type":"ContainerStarted","Data":"9536f0c286e2c59c01c379ba50cbd1648335710de97cab3351bd58120fc4fb87"} Jul 01 19:22:11.768386 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.768335 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-nrds2" podStartSLOduration=2.401985915 podStartE2EDuration="4.768315231s" totalImagesPullingTime="2.366329316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:11.767838781 +0000 UTC m=+43.880387380" watchObservedRunningTime="2026-07-01 19:22:11.768315231 +0000 UTC m=+43.880863831" Jul 01 19:22:11.796040 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:11.795965 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbh8p" podStartSLOduration=1.8955517899999998 podStartE2EDuration="4.795947825s" totalImagesPullingTime="2.900396035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:11.79479491 +0000 UTC m=+43.907343508" watchObservedRunningTime="2026-07-01 19:22:11.795947825 +0000 UTC m=+43.908496424" Jul 01 19:22:12.014862 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:12.014793 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d5cbr_0f3bf7f8-6c2b-4600-a959-65d0bbd34195/node-ca/0.log" Jul 01 19:22:12.227662 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:12.227601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:12.227662 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:12.227673 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:12.227920 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:12.227783 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:22:12.227920 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:12.227859 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:16.227839961 +0000 UTC m=+48.340388551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : secret "insights-runtime-extractor-tls" not found Jul 01 19:22:12.227920 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:12.227792 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:22:12.227920 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:12.227918 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:16.227905309 +0000 UTC m=+48.340453885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : configmap "kube-rbac-proxy" not found Jul 01 19:22:13.013847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:13.013790 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbh8p_909f7802-3f30-4305-888f-7d688cd0e003/serve-healthcheck-canary/0.log" Jul 01 19:22:13.413383 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:13.413345 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-nrds2_d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f/migrator/0.log" Jul 01 19:22:13.614242 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:13.614221 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-nrds2_d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f/graceful-termination/0.log" Jul 01 19:22:13.757757 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:13.757670 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" event={"ID":"503c7647-a1dd-45c8-8574-c5e09b8426ac","Type":"ContainerStarted","Data":"ee4bb17c4a3abda80cf2d7633ba62b1c07020416107818398d8fd8a997ded82a"} Jul 01 19:22:13.780866 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:13.778724 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-klwbj" podStartSLOduration=1.049044587 podStartE2EDuration="3.77870056s" totalImagesPullingTime="2.729655973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:13.776509664 +0000 UTC m=+45.889058261" watchObservedRunningTime="2026-07-01 19:22:13.77870056 +0000 UTC m=+45.891249159" Jul 01 19:22:15.052481 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:15.052442 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:15.053149 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:15.052610 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 19:22:15.053149 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:15.052714 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls podName:d415df21-c25d-439a-8147-9d82e9c13079 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:23.052691057 +0000 UTC m=+55.165239647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls") pod "dns-default-g9r8t" (UID: "d415df21-c25d-439a-8147-9d82e9c13079") : secret "dns-default-metrics-tls" not found Jul 01 19:22:16.263033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:16.262843 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:16.263033 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:16.263017 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:22:16.263472 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:16.263055 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:16.263472 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:16.263111 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:24.263088518 +0000 UTC m=+56.375637102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : secret "insights-runtime-extractor-tls" not found Jul 01 19:22:16.263472 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:16.263147 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 19:22:16.263472 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:16.263191 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:24.263180363 +0000 UTC m=+56.375728941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : configmap "kube-rbac-proxy" not found Jul 01 19:22:23.115048 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:23.115011 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:23.118551 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:23.118528 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d415df21-c25d-439a-8147-9d82e9c13079-metrics-tls\") pod \"dns-default-g9r8t\" (UID: \"d415df21-c25d-439a-8147-9d82e9c13079\") " pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:23.191764 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:23.191724 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:23.316714 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:23.316658 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g9r8t"] Jul 01 19:22:23.319027 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:23.318998 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd415df21_c25d_439a_8147_9d82e9c13079.slice/crio-f579c6a72101e3effdc91a7cdcff6d480895153f1801ba914edd54b7245d53ad WatchSource:0}: Error finding container f579c6a72101e3effdc91a7cdcff6d480895153f1801ba914edd54b7245d53ad: Status 404 returned error can't find the container with id f579c6a72101e3effdc91a7cdcff6d480895153f1801ba914edd54b7245d53ad Jul 01 19:22:23.784343 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:23.784313 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g9r8t" event={"ID":"d415df21-c25d-439a-8147-9d82e9c13079","Type":"ContainerStarted","Data":"f579c6a72101e3effdc91a7cdcff6d480895153f1801ba914edd54b7245d53ad"} Jul 01 19:22:24.325833 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:24.325796 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:24.326293 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:24.325858 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:24.326293 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:24.325947 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 19:22:24.326293 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:24.326014 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls podName:8ebd2222-c969-4a62-a631-4a059587a11e nodeName:}" failed. No retries permitted until 2026-07-01 19:22:40.325995296 +0000 UTC m=+72.438543884 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzdzg" (UID: "8ebd2222-c969-4a62-a631-4a059587a11e") : secret "insights-runtime-extractor-tls" not found Jul 01 19:22:24.326513 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:24.326481 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ebd2222-c969-4a62-a631-4a059587a11e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:25.795467 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:25.795431 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g9r8t" event={"ID":"d415df21-c25d-439a-8147-9d82e9c13079","Type":"ContainerStarted","Data":"7a594f538e4e2f815f43d107e7ac913febfa66a8d549652a9b01cfd6b54511d2"} Jul 01 19:22:25.795865 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:25.795511 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g9r8t" event={"ID":"d415df21-c25d-439a-8147-9d82e9c13079","Type":"ContainerStarted","Data":"b71f00a00264fcfb673ab43221aaf4216f64f4797e8616a4b69d522da4651ad0"} Jul 01 19:22:25.795865 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:25.795553 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:25.814876 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:25.814834 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g9r8t" podStartSLOduration=17.179828899 podStartE2EDuration="18.81482095s" totalImagesPullingTime="1.634992051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:25.814481818 +0000 UTC m=+57.927030416" watchObservedRunningTime="2026-07-01 19:22:25.81482095 +0000 UTC m=+57.927369548" Jul 01 19:22:31.677442 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.677411 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2"] Jul 01 19:22:31.680808 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.680791 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.683817 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.683793 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6zz75\"" Jul 01 19:22:31.683929 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.683815 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 01 19:22:31.684415 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.684400 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 19:22:31.686926 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.686601 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 19:22:31.703779 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.703750 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 19:22:31.716690 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.716663 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-27vrc" Jul 01 19:22:31.736466 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.736441 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2"] Jul 01 19:22:31.781750 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.781717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrhxp\" (UniqueName: \"kubernetes.io/projected/81f77585-daf5-4bbd-bda8-93e0d5957daf-kube-api-access-wrhxp\") pod \"managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2\" (UID: \"81f77585-daf5-4bbd-bda8-93e0d5957daf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.781912 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.781782 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/81f77585-daf5-4bbd-bda8-93e0d5957daf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2\" (UID: \"81f77585-daf5-4bbd-bda8-93e0d5957daf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.877899 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.877867 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54b569d4b9-trr5h"] Jul 01 19:22:31.879663 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.879647 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.882254 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.882230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/81f77585-daf5-4bbd-bda8-93e0d5957daf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2\" (UID: \"81f77585-daf5-4bbd-bda8-93e0d5957daf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.882361 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.882339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrhxp\" (UniqueName: \"kubernetes.io/projected/81f77585-daf5-4bbd-bda8-93e0d5957daf-kube-api-access-wrhxp\") pod \"managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2\" (UID: \"81f77585-daf5-4bbd-bda8-93e0d5957daf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.882596 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.882572 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 19:22:31.884843 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.884826 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 19:22:31.884947 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.884870 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/81f77585-daf5-4bbd-bda8-93e0d5957daf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2\" (UID: \"81f77585-daf5-4bbd-bda8-93e0d5957daf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.897146 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.897122 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 19:22:31.897250 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.897190 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wklt5\"" Jul 01 19:22:31.912150 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.911040 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 19:22:31.919101 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.919075 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54b569d4b9-trr5h"] Jul 01 19:22:31.922877 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.922850 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrhxp\" (UniqueName: \"kubernetes.io/projected/81f77585-daf5-4bbd-bda8-93e0d5957daf-kube-api-access-wrhxp\") pod \"managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2\" (UID: \"81f77585-daf5-4bbd-bda8-93e0d5957daf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:31.983086 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.982992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/332c5816-d00f-46a0-89f7-ab17cff8db95-trusted-ca\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983086 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983030 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/332c5816-d00f-46a0-89f7-ab17cff8db95-installation-pull-secrets\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983086 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/332c5816-d00f-46a0-89f7-ab17cff8db95-image-registry-private-configuration\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983143 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/332c5816-d00f-46a0-89f7-ab17cff8db95-ca-trust-extracted\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983189 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-registry-tls\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983214 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/332c5816-d00f-46a0-89f7-ab17cff8db95-registry-certificates\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983246 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj42f\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-kube-api-access-bj42f\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.983297 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.983292 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-bound-sa-token\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:31.999058 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:31.999033 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" Jul 01 19:22:32.083967 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.083937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-bound-sa-token\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084126 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.083991 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/332c5816-d00f-46a0-89f7-ab17cff8db95-trusted-ca\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084126 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084016 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/332c5816-d00f-46a0-89f7-ab17cff8db95-installation-pull-secrets\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084126 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084066 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/332c5816-d00f-46a0-89f7-ab17cff8db95-image-registry-private-configuration\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084126 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084102 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/332c5816-d00f-46a0-89f7-ab17cff8db95-ca-trust-extracted\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084336 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084138 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-registry-tls\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084336 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084173 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/332c5816-d00f-46a0-89f7-ab17cff8db95-registry-certificates\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084336 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084201 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj42f\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-kube-api-access-bj42f\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.084897 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.084862 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/332c5816-d00f-46a0-89f7-ab17cff8db95-ca-trust-extracted\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.085156 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.085131 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/332c5816-d00f-46a0-89f7-ab17cff8db95-registry-certificates\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.085336 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.085314 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/332c5816-d00f-46a0-89f7-ab17cff8db95-trusted-ca\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.086735 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.086714 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/332c5816-d00f-46a0-89f7-ab17cff8db95-image-registry-private-configuration\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.086844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.086781 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/332c5816-d00f-46a0-89f7-ab17cff8db95-installation-pull-secrets\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.087891 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.087865 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-registry-tls\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.098973 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.098947 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-bound-sa-token\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.099521 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.099496 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj42f\" (UniqueName: \"kubernetes.io/projected/332c5816-d00f-46a0-89f7-ab17cff8db95-kube-api-access-bj42f\") pod \"image-registry-54b569d4b9-trr5h\" (UID: \"332c5816-d00f-46a0-89f7-ab17cff8db95\") " pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.142723 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.142691 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2"] Jul 01 19:22:32.146263 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:32.146234 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81f77585_daf5_4bbd_bda8_93e0d5957daf.slice/crio-19faedcc318e537043c7e83fc6048e882e2a389165098568368b4785ffcfba6a WatchSource:0}: Error finding container 19faedcc318e537043c7e83fc6048e882e2a389165098568368b4785ffcfba6a: Status 404 returned error can't find the container with id 19faedcc318e537043c7e83fc6048e882e2a389165098568368b4785ffcfba6a Jul 01 19:22:32.193144 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.193105 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.327176 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.327118 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54b569d4b9-trr5h"] Jul 01 19:22:32.329516 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:32.329490 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332c5816_d00f_46a0_89f7_ab17cff8db95.slice/crio-b5c0612a1e331a0b8cede1eadf8596fe71c2426efc227df125501dd80959c7ab WatchSource:0}: Error finding container b5c0612a1e331a0b8cede1eadf8596fe71c2426efc227df125501dd80959c7ab: Status 404 returned error can't find the container with id b5c0612a1e331a0b8cede1eadf8596fe71c2426efc227df125501dd80959c7ab Jul 01 19:22:32.817590 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.817550 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" event={"ID":"332c5816-d00f-46a0-89f7-ab17cff8db95","Type":"ContainerStarted","Data":"13c05b4d8d15af1a866d38cbd5aeaaa680dca103e9f4c1958f63339408af321e"} Jul 01 19:22:32.818052 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.817603 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" event={"ID":"332c5816-d00f-46a0-89f7-ab17cff8db95","Type":"ContainerStarted","Data":"b5c0612a1e331a0b8cede1eadf8596fe71c2426efc227df125501dd80959c7ab"} Jul 01 19:22:32.818052 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.817664 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:32.818712 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.818691 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" event={"ID":"81f77585-daf5-4bbd-bda8-93e0d5957daf","Type":"ContainerStarted","Data":"19faedcc318e537043c7e83fc6048e882e2a389165098568368b4785ffcfba6a"} Jul 01 19:22:32.856867 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:32.856817 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" podStartSLOduration=1.856799494 podStartE2EDuration="1.856799494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:32.855461378 +0000 UTC m=+64.968009978" watchObservedRunningTime="2026-07-01 19:22:32.856799494 +0000 UTC m=+64.969348091" Jul 01 19:22:34.102528 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.102492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:34.105497 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.105473 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 19:22:34.115506 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.115478 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622a52e8-4365-440e-8a21-3c13ff32a70e-metrics-certs\") pod \"network-metrics-daemon-d5xdw\" (UID: \"622a52e8-4365-440e-8a21-3c13ff32a70e\") " pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:34.202849 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.202813 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:34.203016 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.202861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:34.206669 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.206649 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 19:22:34.206669 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.206665 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 19:22:34.216258 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.216232 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4761df2-8e35-4f4a-a6f7-20363c365ca4-original-pull-secret\") pod \"global-pull-secret-syncer-vnj75\" (UID: \"a4761df2-8e35-4f4a-a6f7-20363c365ca4\") " pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:34.216948 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.216928 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 19:22:34.226399 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.226373 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t4r8\" (UniqueName: \"kubernetes.io/projected/be8a31ef-4a82-4e5b-9f13-934033eb47a3-kube-api-access-8t4r8\") pod \"network-check-target-j4rjt\" (UID: \"be8a31ef-4a82-4e5b-9f13-934033eb47a3\") " pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:34.368408 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.368334 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zbvbl\"" Jul 01 19:22:34.371069 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.371048 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vnj75" Jul 01 19:22:34.376252 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.376221 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:34.378637 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.378606 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mzk4c\"" Jul 01 19:22:34.387160 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:34.387123 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5xdw" Jul 01 19:22:35.076525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.076493 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d5xdw"] Jul 01 19:22:35.092927 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.092892 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vnj75"] Jul 01 19:22:35.117041 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.117013 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j4rjt"] Jul 01 19:22:35.192722 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:35.192639 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622a52e8_4365_440e_8a21_3c13ff32a70e.slice/crio-84ffa0608eb19c747d7ed05f26972b939c887a03a99fdec7ba39f5aec31d4edc WatchSource:0}: Error finding container 84ffa0608eb19c747d7ed05f26972b939c887a03a99fdec7ba39f5aec31d4edc: Status 404 returned error can't find the container with id 84ffa0608eb19c747d7ed05f26972b939c887a03a99fdec7ba39f5aec31d4edc Jul 01 19:22:35.736584 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.736555 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr"] Jul 01 19:22:35.754010 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.753991 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:35.763465 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.763444 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f8l6r\"" Jul 01 19:22:35.763574 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.763466 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 01 19:22:35.766956 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.766935 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr"] Jul 01 19:22:35.800538 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.800506 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g9r8t" Jul 01 19:22:35.814342 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.814315 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c7224345-e559-47d4-bd8f-bd318d0ad9cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tpcjr\" (UID: \"c7224345-e559-47d4-bd8f-bd318d0ad9cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:35.831216 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.831172 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" event={"ID":"81f77585-daf5-4bbd-bda8-93e0d5957daf","Type":"ContainerStarted","Data":"7f77b5b11f643961096c91bc22f980051f106189dae8ad6b3bacba1289f1314e"} Jul 01 19:22:35.832605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.832577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vnj75" event={"ID":"a4761df2-8e35-4f4a-a6f7-20363c365ca4","Type":"ContainerStarted","Data":"f896e8796bf51d28d21cc972e6916ef6a011a3eed7eefefca0d6481bc29cdae8"} Jul 01 19:22:35.833761 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.833732 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j4rjt" event={"ID":"be8a31ef-4a82-4e5b-9f13-934033eb47a3","Type":"ContainerStarted","Data":"40452b4563f9ab0fb3dc093242311c23f437603b5f37c895f6aa2b61cebd6e92"} Jul 01 19:22:35.835191 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.835156 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d5xdw" event={"ID":"622a52e8-4365-440e-8a21-3c13ff32a70e","Type":"ContainerStarted","Data":"84ffa0608eb19c747d7ed05f26972b939c887a03a99fdec7ba39f5aec31d4edc"} Jul 01 19:22:35.886980 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.886917 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d4f848bf6-w9lm2" podStartSLOduration=1.822734517 podStartE2EDuration="4.886895438s" totalImagesPullingTime="3.064160921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:35.886450191 +0000 UTC m=+67.998998792" watchObservedRunningTime="2026-07-01 19:22:35.886895438 +0000 UTC m=+67.999444038" Jul 01 19:22:35.914835 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.914799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c7224345-e559-47d4-bd8f-bd318d0ad9cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tpcjr\" (UID: \"c7224345-e559-47d4-bd8f-bd318d0ad9cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:35.918079 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:35.918054 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c7224345-e559-47d4-bd8f-bd318d0ad9cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-tpcjr\" (UID: \"c7224345-e559-47d4-bd8f-bd318d0ad9cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:36.062298 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:36.062252 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:36.442139 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:36.442063 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr"] Jul 01 19:22:36.448129 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:36.448098 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7224345_e559_47d4_bd8f_bd318d0ad9cd.slice/crio-7706fcdf5462251cdda68899d0fdea83c4e77285620d885630a9960290b3626e WatchSource:0}: Error finding container 7706fcdf5462251cdda68899d0fdea83c4e77285620d885630a9960290b3626e: Status 404 returned error can't find the container with id 7706fcdf5462251cdda68899d0fdea83c4e77285620d885630a9960290b3626e Jul 01 19:22:36.842123 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:36.842028 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d5xdw" event={"ID":"622a52e8-4365-440e-8a21-3c13ff32a70e","Type":"ContainerStarted","Data":"36ec9c8a372e5939e1125600e1078605d2f896399b087d4a53a539fa8d29090e"} Jul 01 19:22:36.842123 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:36.842088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d5xdw" event={"ID":"622a52e8-4365-440e-8a21-3c13ff32a70e","Type":"ContainerStarted","Data":"9192e365498d86e355bc969451b24ac0f69033635f608376952893b2963b8490"} Jul 01 19:22:36.843924 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:36.843892 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" event={"ID":"c7224345-e559-47d4-bd8f-bd318d0ad9cd","Type":"ContainerStarted","Data":"7706fcdf5462251cdda68899d0fdea83c4e77285620d885630a9960290b3626e"} Jul 01 19:22:36.860714 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:36.860655 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d5xdw" podStartSLOduration=66.746094264 podStartE2EDuration="1m7.860615014s" totalImagesPullingTime="1.11452075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:36.859999466 +0000 UTC m=+68.972548068" watchObservedRunningTime="2026-07-01 19:22:36.860615014 +0000 UTC m=+68.973163615" Jul 01 19:22:40.352693 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.352659 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:40.355765 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.355170 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ebd2222-c969-4a62-a631-4a059587a11e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzdzg\" (UID: \"8ebd2222-c969-4a62-a631-4a059587a11e\") " pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:40.595598 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.595570 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tz26f\"" Jul 01 19:22:40.602834 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.602809 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zzdzg" Jul 01 19:22:40.727405 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.727373 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zzdzg"] Jul 01 19:22:40.730772 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:40.730747 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ebd2222_c969_4a62_a631_4a059587a11e.slice/crio-7c4c5655056e93379e4c6f55616c5cff3d1cabe231898b76c0cf3414ec752e98 WatchSource:0}: Error finding container 7c4c5655056e93379e4c6f55616c5cff3d1cabe231898b76c0cf3414ec752e98: Status 404 returned error can't find the container with id 7c4c5655056e93379e4c6f55616c5cff3d1cabe231898b76c0cf3414ec752e98 Jul 01 19:22:40.860806 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.860708 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" event={"ID":"c7224345-e559-47d4-bd8f-bd318d0ad9cd","Type":"ContainerStarted","Data":"f6d06012754bf71a53e19b8f49f76a23459b104b4e08fd72bc9aefb48695c6d8"} Jul 01 19:22:40.860806 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.860764 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:40.861990 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.861956 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzdzg" event={"ID":"8ebd2222-c969-4a62-a631-4a059587a11e","Type":"ContainerStarted","Data":"7c4c5655056e93379e4c6f55616c5cff3d1cabe231898b76c0cf3414ec752e98"} Jul 01 19:22:40.863520 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.863491 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vnj75" event={"ID":"a4761df2-8e35-4f4a-a6f7-20363c365ca4","Type":"ContainerStarted","Data":"dd78efbd827c15e4caec2c9ce0e068d565c31a9dfcc8f711ffd9e5ea1aece80c"} Jul 01 19:22:40.865108 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.865084 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j4rjt" event={"ID":"be8a31ef-4a82-4e5b-9f13-934033eb47a3","Type":"ContainerStarted","Data":"d579c6b2b15d8e5d7ee09d0274869cc38938267c545ca76d1ea3742e3da83799"} Jul 01 19:22:40.865229 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.865217 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:22:40.866147 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.866127 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" Jul 01 19:22:40.881760 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.881697 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-tpcjr" podStartSLOduration=2.088081085 podStartE2EDuration="5.881681781s" totalImagesPullingTime="3.793600696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:40.880876439 +0000 UTC m=+72.993425041" watchObservedRunningTime="2026-07-01 19:22:40.881681781 +0000 UTC m=+72.994230379" Jul 01 19:22:40.936299 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.936246 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j4rjt" podStartSLOduration=66.89556019 podStartE2EDuration="1m11.936228428s" totalImagesPullingTime="5.040668238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:40.904650489 +0000 UTC m=+73.017199093" watchObservedRunningTime="2026-07-01 19:22:40.936228428 +0000 UTC m=+73.048777026" Jul 01 19:22:40.984851 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:40.984797 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vnj75" podStartSLOduration=66.929658715 podStartE2EDuration="1m11.98477827s" totalImagesPullingTime="5.055119555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:40.983492653 +0000 UTC m=+73.096041251" watchObservedRunningTime="2026-07-01 19:22:40.98477827 +0000 UTC m=+73.097326868" Jul 01 19:22:45.884918 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:45.884884 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzdzg" event={"ID":"8ebd2222-c969-4a62-a631-4a059587a11e","Type":"ContainerStarted","Data":"6925f480a911e7d7746902148fdb29d6b0b2dcd5ce7204286b158588b5da990f"} Jul 01 19:22:45.885306 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:45.884951 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzdzg" event={"ID":"8ebd2222-c969-4a62-a631-4a059587a11e","Type":"ContainerStarted","Data":"8fabae5dcce1e60d437fdbdb97715d7251b9e9c4c2e56bbcce61dd5477fea011"} Jul 01 19:22:48.199323 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.199277 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj"] Jul 01 19:22:48.210142 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.210085 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.213857 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.213826 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj"] Jul 01 19:22:48.214341 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.214314 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mjjlw\"" Jul 01 19:22:48.214341 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.214339 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 01 19:22:48.214601 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.214578 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 19:22:48.214883 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.214811 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 19:22:48.214997 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.214980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 01 19:22:48.215052 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.215009 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 19:22:48.232592 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.232563 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-88f4d"] Jul 01 19:22:48.235951 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.235930 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.239298 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.239274 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k8ffm\"" Jul 01 19:22:48.239454 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.239441 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 19:22:48.239561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.239543 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 19:22:48.239635 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.239543 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 19:22:48.317930 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.317894 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-tls\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318140 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.318333 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.318480 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x7kk\" (UniqueName: \"kubernetes.io/projected/dd537fa1-b864-433e-8546-be82b2614c99-kube-api-access-2x7kk\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.318601 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318601 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318557 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-sys\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318601 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-textfile\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318886 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318642 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd537fa1-b864-433e-8546-be82b2614c99-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.318886 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-root\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318886 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318709 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-wtmp\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318886 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f25080-984f-4295-85e1-68a9fae8fbb7-metrics-client-ca\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318886 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.318886 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.318831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-794rl\" (UniqueName: \"kubernetes.io/projected/67f25080-984f-4295-85e1-68a9fae8fbb7-kube-api-access-794rl\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419329 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419295 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419337 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419364 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x7kk\" (UniqueName: \"kubernetes.io/projected/dd537fa1-b864-433e-8546-be82b2614c99-kube-api-access-2x7kk\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419389 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-sys\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-textfile\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd537fa1-b864-433e-8546-be82b2614c99-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.419511 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-root\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419542 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-wtmp\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f25080-984f-4295-85e1-68a9fae8fbb7-metrics-client-ca\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419602 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-794rl\" (UniqueName: \"kubernetes.io/projected/67f25080-984f-4295-85e1-68a9fae8fbb7-kube-api-access-794rl\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.419688 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-tls\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:48.419815 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 01 19:22:48.419934 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:48.419881 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-tls podName:67f25080-984f-4295-85e1-68a9fae8fbb7 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:48.919860526 +0000 UTC m=+81.032409118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-tls") pod "node-exporter-88f4d" (UID: "67f25080-984f-4295-85e1-68a9fae8fbb7") : secret "node-exporter-tls" not found Jul 01 19:22:48.420268 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.420199 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-root\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.420344 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-wtmp\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.420409 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd537fa1-b864-433e-8546-be82b2614c99-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:48.420423 2570 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:22:48.420481 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-tls podName:dd537fa1-b864-433e-8546-be82b2614c99 nodeName:}" failed. No retries permitted until 2026-07-01 19:22:48.920463329 +0000 UTC m=+81.033011911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-tls") pod "openshift-state-metrics-57f99c8b49-nsnbj" (UID: "dd537fa1-b864-433e-8546-be82b2614c99") : secret "openshift-state-metrics-tls" not found Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.420478 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67f25080-984f-4295-85e1-68a9fae8fbb7-sys\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.420745 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-textfile\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.421042 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.420937 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f25080-984f-4295-85e1-68a9fae8fbb7-metrics-client-ca\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.421553 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.421234 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.423253 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.423227 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.423477 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.423454 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.433784 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.433755 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-794rl\" (UniqueName: \"kubernetes.io/projected/67f25080-984f-4295-85e1-68a9fae8fbb7-kube-api-access-794rl\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.440107 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.440082 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x7kk\" (UniqueName: \"kubernetes.io/projected/dd537fa1-b864-433e-8546-be82b2614c99-kube-api-access-2x7kk\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.898213 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.898109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzdzg" event={"ID":"8ebd2222-c969-4a62-a631-4a059587a11e","Type":"ContainerStarted","Data":"4fcbf8e064a7deed514511db720953b74563905039ff6cd8d6429de611c3c27f"} Jul 01 19:22:48.925242 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.925207 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-tls\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.925242 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.925247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:48.927536 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.927512 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67f25080-984f-4295-85e1-68a9fae8fbb7-node-exporter-tls\") pod \"node-exporter-88f4d\" (UID: \"67f25080-984f-4295-85e1-68a9fae8fbb7\") " pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:48.927660 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:48.927597 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd537fa1-b864-433e-8546-be82b2614c99-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-nsnbj\" (UID: \"dd537fa1-b864-433e-8546-be82b2614c99\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:49.125779 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.125741 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" Jul 01 19:22:49.147774 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.147750 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88f4d" Jul 01 19:22:49.157834 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:49.157803 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f25080_984f_4295_85e1_68a9fae8fbb7.slice/crio-3ca48b2e96e08cf004bb1ef98e379dd437e1d85a47ac8da4384d000ecc9c2978 WatchSource:0}: Error finding container 3ca48b2e96e08cf004bb1ef98e379dd437e1d85a47ac8da4384d000ecc9c2978: Status 404 returned error can't find the container with id 3ca48b2e96e08cf004bb1ef98e379dd437e1d85a47ac8da4384d000ecc9c2978 Jul 01 19:22:49.270840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.270801 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zzdzg" podStartSLOduration=33.593319741 podStartE2EDuration="41.270781841s" totalImagesPullingTime="7.6774621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:22:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:48.956981137 +0000 UTC m=+81.069529732" watchObservedRunningTime="2026-07-01 19:22:49.270781841 +0000 UTC m=+81.383330440" Jul 01 19:22:49.271381 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.271365 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj"] Jul 01 19:22:49.272739 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:49.272714 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd537fa1_b864_433e_8546_be82b2614c99.slice/crio-42a8b50bfc7dd0cea177b852b98d86a4e1ecbc64c779dab0f5a82d89a3313769 WatchSource:0}: Error finding container 42a8b50bfc7dd0cea177b852b98d86a4e1ecbc64c779dab0f5a82d89a3313769: Status 404 returned error can't find the container with id 42a8b50bfc7dd0cea177b852b98d86a4e1ecbc64c779dab0f5a82d89a3313769 Jul 01 19:22:49.909282 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.909241 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" event={"ID":"dd537fa1-b864-433e-8546-be82b2614c99","Type":"ContainerStarted","Data":"544dcf2d662d3e822f234b6e9a85ca6da4bf007f63a14e4031e77dc9a5318fbe"} Jul 01 19:22:49.909462 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.909347 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" event={"ID":"dd537fa1-b864-433e-8546-be82b2614c99","Type":"ContainerStarted","Data":"be00ba53487c970c6668163b683c13e08d60f0d4cd0d1857e61e9cfb7c5bd377"} Jul 01 19:22:49.909462 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.909380 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" event={"ID":"dd537fa1-b864-433e-8546-be82b2614c99","Type":"ContainerStarted","Data":"42a8b50bfc7dd0cea177b852b98d86a4e1ecbc64c779dab0f5a82d89a3313769"} Jul 01 19:22:49.910616 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:49.910590 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88f4d" event={"ID":"67f25080-984f-4295-85e1-68a9fae8fbb7","Type":"ContainerStarted","Data":"3ca48b2e96e08cf004bb1ef98e379dd437e1d85a47ac8da4384d000ecc9c2978"} Jul 01 19:22:50.915998 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:50.915964 2570 generic.go:356] "Generic (PLEG): container finished" podID="67f25080-984f-4295-85e1-68a9fae8fbb7" containerID="d5bd4d5d839d1260340637b6b9622fb94ff9e5cdd0ec42e7045b0293e846f6e0" exitCode=0 Jul 01 19:22:50.916461 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:50.916058 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88f4d" event={"ID":"67f25080-984f-4295-85e1-68a9fae8fbb7","Type":"ContainerDied","Data":"d5bd4d5d839d1260340637b6b9622fb94ff9e5cdd0ec42e7045b0293e846f6e0"} Jul 01 19:22:50.918160 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:50.918132 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" event={"ID":"dd537fa1-b864-433e-8546-be82b2614c99","Type":"ContainerStarted","Data":"bceece1241b95c5293fec60cd88a32e1e7e42153c2a39d67d3373694add1deeb"} Jul 01 19:22:51.024147 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.024100 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-nsnbj" podStartSLOduration=2.037132378 podStartE2EDuration="3.02407915s" totalImagesPullingTime="986.946772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:51.023309167 +0000 UTC m=+83.135857777" watchObservedRunningTime="2026-07-01 19:22:51.02407915 +0000 UTC m=+83.136627751" Jul 01 19:22:51.764812 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.764777 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9"] Jul 01 19:22:51.768376 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.768358 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.777167 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.777149 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 19:22:51.778176 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.778157 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a99qijcj6ksnn\"" Jul 01 19:22:51.778284 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.778236 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fqmg9\"" Jul 01 19:22:51.778284 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.778248 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 19:22:51.778284 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.778270 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 19:22:51.778399 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.778297 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 19:22:51.778978 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.778964 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 19:22:51.812436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.812406 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9"] Jul 01 19:22:51.851379 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851351 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851534 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-grpc-tls\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851534 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-tls\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851534 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851467 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw96f\" (UniqueName: \"kubernetes.io/projected/578f85e0-94fc-471d-b722-9b8458e824df-kube-api-access-rw96f\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851534 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851498 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851721 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851534 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851721 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.851721 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.851695 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/578f85e0-94fc-471d-b722-9b8458e824df-metrics-client-ca\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.924781 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.924742 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88f4d" event={"ID":"67f25080-984f-4295-85e1-68a9fae8fbb7","Type":"ContainerStarted","Data":"00f7f2a54d92284b1fa53c4a85f43de9f3237500bd5c8727331dcbd1cbb4b4b0"} Jul 01 19:22:51.925189 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.924812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88f4d" event={"ID":"67f25080-984f-4295-85e1-68a9fae8fbb7","Type":"ContainerStarted","Data":"568c07788d153498bcda503babdf117f61fb142b77117e05a91d60cb2bee1fdd"} Jul 01 19:22:51.952755 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.952717 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.952910 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.952764 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-grpc-tls\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.952957 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.952943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-tls\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.953059 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.952996 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw96f\" (UniqueName: \"kubernetes.io/projected/578f85e0-94fc-471d-b722-9b8458e824df-kube-api-access-rw96f\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.953059 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.953047 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.953154 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.953086 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.953154 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.953133 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.953240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.953174 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/578f85e0-94fc-471d-b722-9b8458e824df-metrics-client-ca\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.954222 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.954174 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/578f85e0-94fc-471d-b722-9b8458e824df-metrics-client-ca\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.955831 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.955785 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.955989 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.955946 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-grpc-tls\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.956366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.956337 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-tls\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.956528 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.956506 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.957056 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.957034 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.957111 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.957044 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/578f85e0-94fc-471d-b722-9b8458e824df-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:51.964251 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:51.964221 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw96f\" (UniqueName: \"kubernetes.io/projected/578f85e0-94fc-471d-b722-9b8458e824df-kube-api-access-rw96f\") pod \"thanos-querier-86dd7bdd8b-n6xl9\" (UID: \"578f85e0-94fc-471d-b722-9b8458e824df\") " pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:52.077362 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:52.077327 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:22:52.260203 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:52.260159 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-88f4d" podStartSLOduration=3.25818679 podStartE2EDuration="4.260139547s" totalImagesPullingTime="929.268658ms" totalInitContainerRuntime="72.684099ms" isStatefulPod=true podCreationTimestamp="2026-07-01 19:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:51.977350377 +0000 UTC m=+84.089898975" watchObservedRunningTime="2026-07-01 19:22:52.260139547 +0000 UTC m=+84.372688146" Jul 01 19:22:52.261081 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:52.261054 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9"] Jul 01 19:22:52.261735 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:52.261707 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578f85e0_94fc_471d_b722_9b8458e824df.slice/crio-ae7569210226baf7e1350e502042f22f6c2f634961874a5cabc43f102562c9d8 WatchSource:0}: Error finding container ae7569210226baf7e1350e502042f22f6c2f634961874a5cabc43f102562c9d8: Status 404 returned error can't find the container with id ae7569210226baf7e1350e502042f22f6c2f634961874a5cabc43f102562c9d8 Jul 01 19:22:52.930872 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:52.930815 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"ae7569210226baf7e1350e502042f22f6c2f634961874a5cabc43f102562c9d8"} Jul 01 19:22:53.188521 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.188438 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p"] Jul 01 19:22:53.191911 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.191885 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.195019 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.194994 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zjdqv\"" Jul 01 19:22:53.195857 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.195833 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 01 19:22:53.264709 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.264673 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmtzv\" (UniqueName: \"kubernetes.io/projected/7531648a-08e8-454c-983c-3da6fab2b2cc-kube-api-access-wmtzv\") pod \"monitoring-plugin-cd8656b8-lxr4p\" (UID: \"7531648a-08e8-454c-983c-3da6fab2b2cc\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.264884 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.264765 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7531648a-08e8-454c-983c-3da6fab2b2cc-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-lxr4p\" (UID: \"7531648a-08e8-454c-983c-3da6fab2b2cc\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.366228 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.366196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7531648a-08e8-454c-983c-3da6fab2b2cc-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-lxr4p\" (UID: \"7531648a-08e8-454c-983c-3da6fab2b2cc\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.366407 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.366244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmtzv\" (UniqueName: \"kubernetes.io/projected/7531648a-08e8-454c-983c-3da6fab2b2cc-kube-api-access-wmtzv\") pod \"monitoring-plugin-cd8656b8-lxr4p\" (UID: \"7531648a-08e8-454c-983c-3da6fab2b2cc\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.369197 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.369166 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7531648a-08e8-454c-983c-3da6fab2b2cc-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-lxr4p\" (UID: \"7531648a-08e8-454c-983c-3da6fab2b2cc\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.416861 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.416828 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmtzv\" (UniqueName: \"kubernetes.io/projected/7531648a-08e8-454c-983c-3da6fab2b2cc-kube-api-access-wmtzv\") pod \"monitoring-plugin-cd8656b8-lxr4p\" (UID: \"7531648a-08e8-454c-983c-3da6fab2b2cc\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.420807 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.420773 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p"] Jul 01 19:22:53.502993 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.502910 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:53.654114 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.654073 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p"] Jul 01 19:22:53.826983 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:53.826955 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54b569d4b9-trr5h" Jul 01 19:22:54.147305 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:54.147237 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7531648a_08e8_454c_983c_3da6fab2b2cc.slice/crio-b3efa227603625b94403c93b1bb548edb3a00472c5156c6c97e6fa879a197d0d WatchSource:0}: Error finding container b3efa227603625b94403c93b1bb548edb3a00472c5156c6c97e6fa879a197d0d: Status 404 returned error can't find the container with id b3efa227603625b94403c93b1bb548edb3a00472c5156c6c97e6fa879a197d0d Jul 01 19:22:54.944495 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:54.944459 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"d2b71703b54e69db12a164218b08b607ea097692394e2b260f027ad4f7549ac6"} Jul 01 19:22:54.944676 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:54.944527 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"987aafc6b9b3384e9303180d9f0cf387f49a3b2e25dd8210fd13928662e3310d"} Jul 01 19:22:54.944676 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:54.944563 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"bb4a6528945a019eef8734bf44cfdf0a3f1d013875557eef59a2312d9bb11860"} Jul 01 19:22:54.945678 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:54.945647 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" event={"ID":"7531648a-08e8-454c-983c-3da6fab2b2cc","Type":"ContainerStarted","Data":"b3efa227603625b94403c93b1bb548edb3a00472c5156c6c97e6fa879a197d0d"} Jul 01 19:22:55.377234 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.377127 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:22:55.380733 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.380716 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.386195 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.386171 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ds3ig4g5o97v\"" Jul 01 19:22:55.386306 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.386211 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 19:22:55.386306 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.386249 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 19:22:55.386403 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.386173 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 19:22:55.386723 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.386702 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 19:22:55.386723 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.386718 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 19:22:55.387563 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.387542 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 19:22:55.387813 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.387789 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 19:22:55.387904 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.387884 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 19:22:55.387976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.387948 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 19:22:55.387976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.387956 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 19:22:55.389072 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.389046 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 19:22:55.389072 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.389067 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 19:22:55.390058 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.390042 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rxhqt\"" Jul 01 19:22:55.391862 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.391845 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 19:22:55.393329 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.393313 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 19:22:55.393706 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.393692 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 19:22:55.416984 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.416952 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:22:55.482923 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.482897 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.482923 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.482930 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483105 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.482947 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483105 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.482964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483105 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483105 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483082 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483231 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483231 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483156 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzkbd\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-kube-api-access-wzkbd\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483231 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483180 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483231 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483231 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483221 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483246 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483267 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483321 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483359 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483474 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483375 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483474 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483389 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483474 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483411 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483474 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.483474 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.483455 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.584900 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.584807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.584900 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.584866 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.584900 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.584894 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585183 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.584937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585183 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585122 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzkbd\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-kube-api-access-wzkbd\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585183 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585170 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585335 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585199 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585335 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585335 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585335 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585301 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585449 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585525 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585503 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585564 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585618 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.585844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.585650 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.586664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.586208 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.586664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.586333 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.586664 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.586565 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.588888 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.587957 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.588888 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.588119 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.588888 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.588402 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.588888 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.588580 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.589682 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.589660 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.590147 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.590124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.590396 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.590374 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.590804 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.590783 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.591004 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.590981 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.591225 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.591170 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.591322 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.591306 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.591734 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.591687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.591826 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.591756 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.591923 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.591902 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.592723 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.592696 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.618246 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.618218 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzkbd\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-kube-api-access-wzkbd\") pod \"prometheus-k8s-0\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.691483 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.691445 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:22:55.930939 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.930915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:22:55.931502 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:22:55.931393 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143804f4_1eeb_42e0_ac43_3e6ba2b6a7d7.slice/crio-c49697be05a4f3cdff7c6b272e5a5ae6b6bd24def8f9d92e037b2876f528d7b6 WatchSource:0}: Error finding container c49697be05a4f3cdff7c6b272e5a5ae6b6bd24def8f9d92e037b2876f528d7b6: Status 404 returned error can't find the container with id c49697be05a4f3cdff7c6b272e5a5ae6b6bd24def8f9d92e037b2876f528d7b6 Jul 01 19:22:55.951572 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.951547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" event={"ID":"7531648a-08e8-454c-983c-3da6fab2b2cc","Type":"ContainerStarted","Data":"b6c5bda8a909320e55d2736980a3dd41a02ce69d97a58b5551b4bc1c9a13c5be"} Jul 01 19:22:55.951787 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.951747 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:55.953052 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.953015 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"c49697be05a4f3cdff7c6b272e5a5ae6b6bd24def8f9d92e037b2876f528d7b6"} Jul 01 19:22:55.957166 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.957146 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" Jul 01 19:22:55.957307 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.957286 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"c92422fb9bf02af1850653a642a821bd3c742b63f1aef95eeb0dd60e3d6ff3c6"} Jul 01 19:22:55.957364 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.957346 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"1f555f8c349d6cd7d305458c698257419277752ad319d50724b1ca8a58a4c2a8"} Jul 01 19:22:55.979231 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:55.979143 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-lxr4p" podStartSLOduration=1.81716532 podStartE2EDuration="2.9791232s" totalImagesPullingTime="1.16195788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:55.978116758 +0000 UTC m=+88.090665357" watchObservedRunningTime="2026-07-01 19:22:55.9791232 +0000 UTC m=+88.091671800" Jul 01 19:22:56.964374 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:56.964344 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" event={"ID":"578f85e0-94fc-471d-b722-9b8458e824df","Type":"ContainerStarted","Data":"b72739ad4a24b70d47aa435fd2f0ea42793c16cb5d2c3b2c16c581d066719816"} Jul 01 19:22:57.014711 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:57.014440 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" podStartSLOduration=2.618918855 podStartE2EDuration="6.014420974s" totalImagesPullingTime="3.395502119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:22:57.01422525 +0000 UTC m=+89.126773849" watchObservedRunningTime="2026-07-01 19:22:57.014420974 +0000 UTC m=+89.126969574" Jul 01 19:22:57.969896 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:57.969863 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" exitCode=0 Jul 01 19:22:57.970340 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:57.969957 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} Jul 01 19:22:57.970340 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:22:57.970180 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:23:01.992276 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:01.992232 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} Jul 01 19:23:01.992649 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:01.992353 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} Jul 01 19:23:01.992649 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:01.992392 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} Jul 01 19:23:01.992649 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:01.992430 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} Jul 01 19:23:01.992649 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:01.992465 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} Jul 01 19:23:01.992649 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:01.992499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerStarted","Data":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} Jul 01 19:23:02.028947 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:02.028889 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.762004985 podStartE2EDuration="7.028872084s" totalImagesPullingTime="4.248196512s" totalInitContainerRuntime="18.670587ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:22:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:23:02.026380742 +0000 UTC m=+94.138929341" watchObservedRunningTime="2026-07-01 19:23:02.028872084 +0000 UTC m=+94.141420682" Jul 01 19:23:03.981047 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:03.981018 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-86dd7bdd8b-n6xl9" Jul 01 19:23:05.692373 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:05.692328 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:23:11.872325 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:11.872297 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j4rjt" Jul 01 19:23:55.692551 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:55.692508 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:23:55.714177 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:55.714149 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:23:56.209872 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:23:56.209844 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:13.765299 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765261 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:24:13.765809 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765549 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy" containerID="cri-o://24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" gracePeriod=600 Jul 01 19:24:13.765809 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765540 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="prometheus" containerID="cri-o://cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" gracePeriod=600 Jul 01 19:24:13.765809 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765615 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" gracePeriod=600 Jul 01 19:24:13.765809 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765658 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-web" containerID="cri-o://35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" gracePeriod=600 Jul 01 19:24:13.765809 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765617 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="thanos-sidecar" containerID="cri-o://cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" gracePeriod=600 Jul 01 19:24:13.765809 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:13.765652 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="config-reloader" containerID="cri-o://46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" gracePeriod=600 Jul 01 19:24:14.023974 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.023950 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.037208 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037184 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-kube-rbac-proxy\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037348 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037218 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-metrics-client-certs\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037348 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037238 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037348 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037271 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-0\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037348 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037296 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037348 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037326 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-grpc-tls\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037358 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-trusted-ca-bundle\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037383 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-2\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037415 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-kubelet-serving-ca-bundle\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037444 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037483 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config-out\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037506 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-1\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037538 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-thanos-prometheus-http-client-file\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037565 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-serving-certs-ca-bundle\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.037605 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037604 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzkbd\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-kube-api-access-wzkbd\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.038048 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037680 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-db\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.038048 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037704 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-tls-assets\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.038048 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037732 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-tls\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.038048 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037757 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-metrics-client-ca\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.038048 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.037798 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-web-config\") pod \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\" (UID: \"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7\") " Jul 01 19:24:14.038530 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.038460 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.038732 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.038704 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.038885 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.038865 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.038948 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.038886 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.039007 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.038979 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.039939 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.039908 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.040850 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.040822 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:24:14.041315 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041127 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.041315 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041202 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.041315 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041281 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.041587 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041555 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:24:14.041881 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041834 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-kube-api-access-wzkbd" (OuterVolumeSpecName: "kube-api-access-wzkbd") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "kube-api-access-wzkbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:24:14.041963 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041915 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:24:14.041963 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041938 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.042067 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.041957 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.042125 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.042095 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config-out" (OuterVolumeSpecName: "config-out") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:24:14.042273 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.042257 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.042494 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.042480 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.042551 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.042538 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config" (OuterVolumeSpecName: "config") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.051398 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.051372 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-web-config" (OuterVolumeSpecName: "web-config") pod "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" (UID: "143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:24:14.138561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138527 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138555 2570 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138566 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-config-out\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138576 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138585 2570 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138595 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138603 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzkbd\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-kube-api-access-wzkbd\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138611 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-db\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138619 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-tls-assets\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138647 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138656 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-configmap-metrics-client-ca\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138684 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-web-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138693 2570 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-kube-rbac-proxy\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138701 2570 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-metrics-client-certs\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138712 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138721 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138729 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138737 2570 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-secret-grpc-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138746 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.138790 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.138753 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:24:14.262845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262814 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" exitCode=0 Jul 01 19:24:14.262845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262837 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" exitCode=0 Jul 01 19:24:14.262845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262843 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" exitCode=0 Jul 01 19:24:14.262845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262851 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" exitCode=0 Jul 01 19:24:14.262845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262856 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" exitCode=0 Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262860 2570 generic.go:356] "Generic (PLEG): container finished" podID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" exitCode=0 Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262894 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262896 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.262987 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.263014 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.263040 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.263062 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.263072 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.263099 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} Jul 01 19:24:14.263133 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.263128 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7","Type":"ContainerDied","Data":"c49697be05a4f3cdff7c6b272e5a5ae6b6bd24def8f9d92e037b2876f528d7b6"} Jul 01 19:24:14.271169 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.271152 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.277998 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.277968 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.284719 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.284705 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.288756 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.288736 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:24:14.291776 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.291757 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.295569 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.295548 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:24:14.298366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.298334 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.305726 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.305711 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.312032 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312015 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.312274 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.312255 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.312329 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312281 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} err="failed to get container status \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" Jul 01 19:24:14.312329 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312313 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.312518 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.312503 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.312564 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312521 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} err="failed to get container status \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" Jul 01 19:24:14.312564 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312533 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.312750 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.312735 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.312801 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312751 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} err="failed to get container status \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" Jul 01 19:24:14.312801 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312763 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.312968 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.312952 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.313005 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312972 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} err="failed to get container status \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" Jul 01 19:24:14.313005 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.312983 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.313187 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.313172 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.313220 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313190 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} err="failed to get container status \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" Jul 01 19:24:14.313220 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313204 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.313381 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.313367 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.313421 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313384 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} err="failed to get container status \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" Jul 01 19:24:14.313421 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313396 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.313568 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:24:14.313554 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.313609 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313570 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} err="failed to get container status \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" Jul 01 19:24:14.313609 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313580 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.313800 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313779 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} err="failed to get container status \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" Jul 01 19:24:14.313871 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.313802 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.314054 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314037 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} err="failed to get container status \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" Jul 01 19:24:14.314104 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314055 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.314286 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314267 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} err="failed to get container status \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" Jul 01 19:24:14.314341 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314288 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.314499 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314481 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} err="failed to get container status \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" Jul 01 19:24:14.314545 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314500 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.314744 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314724 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} err="failed to get container status \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" Jul 01 19:24:14.314794 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314746 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.314941 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314924 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} err="failed to get container status \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" Jul 01 19:24:14.315012 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.314944 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.315188 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315172 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} err="failed to get container status \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" Jul 01 19:24:14.315234 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315188 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.315362 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315348 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} err="failed to get container status \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" Jul 01 19:24:14.315403 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315362 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.315535 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315522 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} err="failed to get container status \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" Jul 01 19:24:14.315582 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315534 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.315730 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315715 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} err="failed to get container status \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" Jul 01 19:24:14.315774 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315730 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.315942 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315923 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} err="failed to get container status \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" Jul 01 19:24:14.316015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.315943 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.316148 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316133 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} err="failed to get container status \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" Jul 01 19:24:14.316212 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316149 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.316342 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316325 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} err="failed to get container status \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" Jul 01 19:24:14.316405 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316343 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.316552 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316536 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} err="failed to get container status \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" Jul 01 19:24:14.316615 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316552 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.316826 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316808 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} err="failed to get container status \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" Jul 01 19:24:14.316876 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.316827 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.317022 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317004 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} err="failed to get container status \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" Jul 01 19:24:14.317083 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317021 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.317236 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317220 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} err="failed to get container status \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" Jul 01 19:24:14.317309 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317238 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.317454 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317436 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} err="failed to get container status \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" Jul 01 19:24:14.317501 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317455 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.317702 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317684 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} err="failed to get container status \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" Jul 01 19:24:14.317748 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317703 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.317905 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317890 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} err="failed to get container status \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" Jul 01 19:24:14.317952 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.317905 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.318108 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.318074 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} err="failed to get container status \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" Jul 01 19:24:14.318175 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.318119 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.318510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.318481 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} err="failed to get container status \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" Jul 01 19:24:14.318510 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.318510 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.319261 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.318776 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} err="failed to get container status \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" Jul 01 19:24:14.319261 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.318803 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.319261 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.319180 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} err="failed to get container status \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" Jul 01 19:24:14.319261 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.319230 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.319754 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.319733 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} err="failed to get container status \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" Jul 01 19:24:14.319754 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.319753 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.319993 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.319974 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} err="failed to get container status \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" Jul 01 19:24:14.319993 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.319994 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.320249 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320231 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} err="failed to get container status \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" Jul 01 19:24:14.320290 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320251 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.320423 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320406 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:24:14.320477 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320461 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} err="failed to get container status \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" Jul 01 19:24:14.320524 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320483 2570 scope.go:117] "RemoveContainer" containerID="4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8" Jul 01 19:24:14.320720 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320702 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8"} err="failed to get container status \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": rpc error: code = NotFound desc = could not find container \"4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8\": container with ID starting with 4a3da2d7656e3d96941d4f18774c85bf967aedd05d53db0414f328889bee35a8 not found: ID does not exist" Jul 01 19:24:14.320720 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320719 2570 scope.go:117] "RemoveContainer" containerID="24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320733 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="init-config-reloader" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320746 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="init-config-reloader" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320761 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="config-reloader" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320767 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="config-reloader" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320774 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="prometheus" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320779 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="prometheus" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320790 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-web" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320796 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-web" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320806 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320814 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy" Jul 01 19:24:14.320823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320825 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="thanos-sidecar" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320831 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="thanos-sidecar" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320838 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-thanos" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320843 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-thanos" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320893 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="prometheus" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320900 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="config-reloader" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320906 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-web" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320912 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320918 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="kube-rbac-proxy-thanos" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320930 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" containerName="thanos-sidecar" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320937 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e"} err="failed to get container status \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": rpc error: code = NotFound desc = could not find container \"24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e\": container with ID starting with 24f5eea7e1a2b97b10e40946dd4b3159c10c41f566b490f8ffde0f4e88d1084e not found: ID does not exist" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.320950 2570 scope.go:117] "RemoveContainer" containerID="35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321154 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278"} err="failed to get container status \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": rpc error: code = NotFound desc = could not find container \"35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278\": container with ID starting with 35a9440d26e72e92ffac69578044b41a0e10c53875be7d7864b1db98adc10278 not found: ID does not exist" Jul 01 19:24:14.321240 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321177 2570 scope.go:117] "RemoveContainer" containerID="cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a" Jul 01 19:24:14.321727 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321398 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a"} err="failed to get container status \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": rpc error: code = NotFound desc = could not find container \"cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a\": container with ID starting with cd710e89468a1e192546420bca70298128e9bb38245d37bc46e0b5e8d936825a not found: ID does not exist" Jul 01 19:24:14.321727 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321414 2570 scope.go:117] "RemoveContainer" containerID="46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289" Jul 01 19:24:14.321727 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321667 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289"} err="failed to get container status \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": rpc error: code = NotFound desc = could not find container \"46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289\": container with ID starting with 46b81280b6719cc1748e6e9b25e0c7a1c88d0b202d59993d5dfc0984cb197289 not found: ID does not exist" Jul 01 19:24:14.321727 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321682 2570 scope.go:117] "RemoveContainer" containerID="cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596" Jul 01 19:24:14.321883 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321864 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596"} err="failed to get container status \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": rpc error: code = NotFound desc = could not find container \"cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596\": container with ID starting with cd182ca28cbf6e2a1fca4b93f3636c3ecfb31404f6daaaa251d01755bc1f5596 not found: ID does not exist" Jul 01 19:24:14.321883 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.321878 2570 scope.go:117] "RemoveContainer" containerID="20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93" Jul 01 19:24:14.322107 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.322084 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93"} err="failed to get container status \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": rpc error: code = NotFound desc = could not find container \"20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93\": container with ID starting with 20aca354b71874793a2c6ad65790b3b599c29d06114c1e656d6e8999cbed7c93 not found: ID does not exist" Jul 01 19:24:14.326688 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.326671 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.329459 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.329439 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 01 19:24:14.329557 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.329486 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 01 19:24:14.329557 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.329486 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 01 19:24:14.329678 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.329594 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 01 19:24:14.329929 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.329912 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 01 19:24:14.330132 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.330118 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 01 19:24:14.330482 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.330467 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 01 19:24:14.330594 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.330579 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 01 19:24:14.330737 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.330613 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 19:24:14.330737 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.330613 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 01 19:24:14.330737 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.330693 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rxhqt\"" Jul 01 19:24:14.331068 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.331050 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ds3ig4g5o97v\"" Jul 01 19:24:14.331181 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.331088 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 01 19:24:14.331181 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.331138 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 01 19:24:14.331181 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.331151 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 01 19:24:14.334236 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.334219 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 01 19:24:14.336235 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.336217 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 01 19:24:14.340116 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340094 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-web-config\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340195 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340128 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340195 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340300 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340300 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340223 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f7a05c-2620-48c5-bbaa-f0d112b17636-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340300 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340245 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340293 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340462 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340507 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340534 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-config\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340577 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.340661 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340656 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.341067 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340695 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.341067 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f7a05c-2620-48c5-bbaa-f0d112b17636-config-out\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.341067 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340751 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.341067 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmjx8\" (UniqueName: \"kubernetes.io/projected/d3f7a05c-2620-48c5-bbaa-f0d112b17636-kube-api-access-xmjx8\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.341067 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.340796 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.342138 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.342118 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:24:14.360095 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.360073 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7" path="/var/lib/kubelet/pods/143804f4-1eeb-42e0-ac43-3e6ba2b6a7d7/volumes" Jul 01 19:24:14.441545 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-web-config\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.441545 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441548 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.441776 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.441776 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.441846 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441820 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f7a05c-2620-48c5-bbaa-f0d112b17636-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.441876 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441854 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441881 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441910 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.441975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442025 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442053 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-config\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442112 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.442165 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442172 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442200 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f7a05c-2620-48c5-bbaa-f0d112b17636-config-out\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmjx8\" (UniqueName: \"kubernetes.io/projected/d3f7a05c-2620-48c5-bbaa-f0d112b17636-kube-api-access-xmjx8\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442279 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442601 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442601 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.442740 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.443049 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.443156 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.443801 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.444678 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-web-config\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.444697 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.444696 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f7a05c-2620-48c5-bbaa-f0d112b17636-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.445375 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.444719 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.445375 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.444902 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.445375 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.445243 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.445560 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.445532 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.445560 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.445554 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.446117 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.446091 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d3f7a05c-2620-48c5-bbaa-f0d112b17636-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.447035 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.447012 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f7a05c-2620-48c5-bbaa-f0d112b17636-config-out\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.447255 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.447235 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-config\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.447334 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.447243 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.447392 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.447329 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.447425 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.447403 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f7a05c-2620-48c5-bbaa-f0d112b17636-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.453561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.453545 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmjx8\" (UniqueName: \"kubernetes.io/projected/d3f7a05c-2620-48c5-bbaa-f0d112b17636-kube-api-access-xmjx8\") pod \"prometheus-k8s-0\" (UID: \"d3f7a05c-2620-48c5-bbaa-f0d112b17636\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.636891 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.636799 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:24:14.773038 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:14.773006 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 01 19:24:14.775254 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:24:14.775233 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f7a05c_2620_48c5_bbaa_f0d112b17636.slice/crio-7ed04feb326fb116ec7abe015d80476a3b23d3bd8ebd49ad271266de65dcea2a WatchSource:0}: Error finding container 7ed04feb326fb116ec7abe015d80476a3b23d3bd8ebd49ad271266de65dcea2a: Status 404 returned error can't find the container with id 7ed04feb326fb116ec7abe015d80476a3b23d3bd8ebd49ad271266de65dcea2a Jul 01 19:24:15.268789 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:15.268749 2570 generic.go:356] "Generic (PLEG): container finished" podID="d3f7a05c-2620-48c5-bbaa-f0d112b17636" containerID="1baf71a4e181da35600c8e0eac2096d2090789fc046e74bf8e983ed570be47c2" exitCode=0 Jul 01 19:24:15.268965 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:15.268834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerDied","Data":"1baf71a4e181da35600c8e0eac2096d2090789fc046e74bf8e983ed570be47c2"} Jul 01 19:24:15.268965 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:15.268899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"7ed04feb326fb116ec7abe015d80476a3b23d3bd8ebd49ad271266de65dcea2a"} Jul 01 19:24:16.276201 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.276165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"538bc9b04192e5dbaf78c1887d3af163e1f4547e5840769ef6e6c83c3dbfe8bf"} Jul 01 19:24:16.276667 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.276256 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"b1eae2972b8ff9f6206fcf97994a17da104f505ce937a3106b6bcbe3adb01d05"} Jul 01 19:24:16.276667 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.276282 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"da0f5fbd43f0ea10897f39b950f7ef6e8fb711a6c3be3b639ab65d013363e0cc"} Jul 01 19:24:16.276667 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.276306 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"1b13c3ee3322976e2f24f3a1d8c2814c72d16eee559a3c515a329298c47f3d9b"} Jul 01 19:24:16.276667 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.276330 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"43eac3173b18ac7b4c3b778cca8242fe86799e29497c8d593327c67b1e684f84"} Jul 01 19:24:16.276667 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.276354 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f7a05c-2620-48c5-bbaa-f0d112b17636","Type":"ContainerStarted","Data":"570c623fb9469187af241490d9c3ba7bba5e100ecfb20c1daf7d24df8052e968"} Jul 01 19:24:16.306182 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:16.306110 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.286546842 podStartE2EDuration="2.306095211s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.548369ms" isStatefulPod=false podCreationTimestamp="2026-07-01 19:24:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:24:16.303377127 +0000 UTC m=+168.415925738" watchObservedRunningTime="2026-07-01 19:24:16.306095211 +0000 UTC m=+168.418643810" Jul 01 19:24:19.637190 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:24:19.637087 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:25:14.637609 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:25:14.637562 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:25:14.652793 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:25:14.652764 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:25:15.517946 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:25:15.517917 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 01 19:26:28.300928 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:26:28.300872 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:26:28.310197 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:26:28.310168 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:26:28.313420 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:26:28.313399 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 19:29:32.943479 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.943442 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-blw8w"] Jul 01 19:29:32.946463 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.946446 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-blw8w" Jul 01 19:29:32.948998 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.948972 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 01 19:29:32.949458 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.949438 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 19:29:32.950180 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.950165 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 19:29:32.950267 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.950238 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-b84r8\"" Jul 01 19:29:32.955762 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:32.955741 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-blw8w"] Jul 01 19:29:33.066438 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.066408 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkm7f\" (UniqueName: \"kubernetes.io/projected/c34d2cf9-67bf-4735-893b-14c0d08cba70-kube-api-access-nkm7f\") pod \"s3-init-blw8w\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " pod="kserve/s3-init-blw8w" Jul 01 19:29:33.066616 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.066469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c34d2cf9-67bf-4735-893b-14c0d08cba70-model-cache\") pod \"s3-init-blw8w\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " pod="kserve/s3-init-blw8w" Jul 01 19:29:33.167866 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.167823 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkm7f\" (UniqueName: \"kubernetes.io/projected/c34d2cf9-67bf-4735-893b-14c0d08cba70-kube-api-access-nkm7f\") pod \"s3-init-blw8w\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " pod="kserve/s3-init-blw8w" Jul 01 19:29:33.167976 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.167898 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c34d2cf9-67bf-4735-893b-14c0d08cba70-model-cache\") pod \"s3-init-blw8w\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " pod="kserve/s3-init-blw8w" Jul 01 19:29:33.168221 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.168207 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c34d2cf9-67bf-4735-893b-14c0d08cba70-model-cache\") pod \"s3-init-blw8w\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " pod="kserve/s3-init-blw8w" Jul 01 19:29:33.183792 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.183763 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkm7f\" (UniqueName: \"kubernetes.io/projected/c34d2cf9-67bf-4735-893b-14c0d08cba70-kube-api-access-nkm7f\") pod \"s3-init-blw8w\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " pod="kserve/s3-init-blw8w" Jul 01 19:29:33.255844 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.255762 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-blw8w" Jul 01 19:29:33.381934 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.381901 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-blw8w"] Jul 01 19:29:33.387102 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:29:33.387069 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc34d2cf9_67bf_4735_893b_14c0d08cba70.slice/crio-fe78cedc7ab3f73968d268575d69803d9d22bd4f646e5a84d4bc2bfaadf28391 WatchSource:0}: Error finding container fe78cedc7ab3f73968d268575d69803d9d22bd4f646e5a84d4bc2bfaadf28391: Status 404 returned error can't find the container with id fe78cedc7ab3f73968d268575d69803d9d22bd4f646e5a84d4bc2bfaadf28391 Jul 01 19:29:33.389058 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.389041 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 19:29:33.464181 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:33.464151 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-blw8w" event={"ID":"c34d2cf9-67bf-4735-893b-14c0d08cba70","Type":"ContainerStarted","Data":"fe78cedc7ab3f73968d268575d69803d9d22bd4f646e5a84d4bc2bfaadf28391"} Jul 01 19:29:37.481379 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:37.481336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-blw8w" event={"ID":"c34d2cf9-67bf-4735-893b-14c0d08cba70","Type":"ContainerStarted","Data":"afad2c0c5e321f565635f8912628156d6a75edeb16a6bd2b0f219d65dc1bc21d"} Jul 01 19:29:39.489350 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:39.489315 2570 generic.go:356] "Generic (PLEG): container finished" podID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerID="afad2c0c5e321f565635f8912628156d6a75edeb16a6bd2b0f219d65dc1bc21d" exitCode=0 Jul 01 19:29:39.489745 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:39.489386 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-blw8w" event={"ID":"c34d2cf9-67bf-4735-893b-14c0d08cba70","Type":"ContainerDied","Data":"afad2c0c5e321f565635f8912628156d6a75edeb16a6bd2b0f219d65dc1bc21d"} Jul 01 19:29:44.513656 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:44.513552 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-blw8w" event={"ID":"c34d2cf9-67bf-4735-893b-14c0d08cba70","Type":"ContainerStarted","Data":"c45b18570ebfb691197ba1f22750b721b7c47901009fa2ee9641cf27e8313c86"} Jul 01 19:29:44.537548 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:29:44.537487 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-blw8w" podStartSLOduration=2.459885472 podStartE2EDuration="12.537467216s" totalImagesPullingTime="7.993461922s" totalInitContainerRuntime="2.084119822s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:29:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:29:44.537124043 +0000 UTC m=+496.649672643" watchObservedRunningTime="2026-07-01 19:29:44.537467216 +0000 UTC m=+496.650015815" Jul 01 19:30:49.760958 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:49.760923 2570 generic.go:356] "Generic (PLEG): container finished" podID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerID="c45b18570ebfb691197ba1f22750b721b7c47901009fa2ee9641cf27e8313c86" exitCode=0 Jul 01 19:30:49.761365 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:49.760971 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-blw8w" event={"ID":"c34d2cf9-67bf-4735-893b-14c0d08cba70","Type":"ContainerDied","Data":"c45b18570ebfb691197ba1f22750b721b7c47901009fa2ee9641cf27e8313c86"} Jul 01 19:30:50.888720 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:50.888695 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-blw8w" Jul 01 19:30:51.003026 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.002993 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkm7f\" (UniqueName: \"kubernetes.io/projected/c34d2cf9-67bf-4735-893b-14c0d08cba70-kube-api-access-nkm7f\") pod \"c34d2cf9-67bf-4735-893b-14c0d08cba70\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " Jul 01 19:30:51.003205 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.003055 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c34d2cf9-67bf-4735-893b-14c0d08cba70-model-cache\") pod \"c34d2cf9-67bf-4735-893b-14c0d08cba70\" (UID: \"c34d2cf9-67bf-4735-893b-14c0d08cba70\") " Jul 01 19:30:51.005198 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.005162 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c34d2cf9-67bf-4735-893b-14c0d08cba70-kube-api-access-nkm7f" (OuterVolumeSpecName: "kube-api-access-nkm7f") pod "c34d2cf9-67bf-4735-893b-14c0d08cba70" (UID: "c34d2cf9-67bf-4735-893b-14c0d08cba70"). InnerVolumeSpecName "kube-api-access-nkm7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:30:51.021578 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.021549 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c34d2cf9-67bf-4735-893b-14c0d08cba70-model-cache" (OuterVolumeSpecName: "model-cache") pod "c34d2cf9-67bf-4735-893b-14c0d08cba70" (UID: "c34d2cf9-67bf-4735-893b-14c0d08cba70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:30:51.103636 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.103597 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkm7f\" (UniqueName: \"kubernetes.io/projected/c34d2cf9-67bf-4735-893b-14c0d08cba70-kube-api-access-nkm7f\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:30:51.103799 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.103655 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c34d2cf9-67bf-4735-893b-14c0d08cba70-model-cache\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:30:51.772447 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.772415 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-blw8w" event={"ID":"c34d2cf9-67bf-4735-893b-14c0d08cba70","Type":"ContainerDied","Data":"fe78cedc7ab3f73968d268575d69803d9d22bd4f646e5a84d4bc2bfaadf28391"} Jul 01 19:30:51.772447 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.772451 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe78cedc7ab3f73968d268575d69803d9d22bd4f646e5a84d4bc2bfaadf28391" Jul 01 19:30:51.772669 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:30:51.772457 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-blw8w" Jul 01 19:31:03.704170 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.704134 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w"] Jul 01 19:31:03.704672 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.704433 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerName="download-hf-model" Jul 01 19:31:03.704672 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.704444 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerName="download-hf-model" Jul 01 19:31:03.704672 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.704454 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerName="s3-init" Jul 01 19:31:03.704672 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.704459 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerName="s3-init" Jul 01 19:31:03.704672 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.704519 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c34d2cf9-67bf-4735-893b-14c0d08cba70" containerName="s3-init" Jul 01 19:31:03.707672 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.707650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.716702 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.716672 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 01 19:31:03.716936 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.716723 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 01 19:31:03.716936 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.716732 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 01 19:31:03.717519 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.717497 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-xs8vf\"" Jul 01 19:31:03.717707 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.717500 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 19:31:03.717843 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.717509 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 01 19:31:03.718123 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.718106 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 01 19:31:03.720153 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.720127 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w"] Jul 01 19:31:03.824031 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.823997 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b355a727-3563-42f4-825e-b18cfe2c9369-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.824222 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.824089 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b355a727-3563-42f4-825e-b18cfe2c9369-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.824222 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.824113 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.824222 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.824141 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xq2j\" (UniqueName: \"kubernetes.io/projected/b355a727-3563-42f4-825e-b18cfe2c9369-kube-api-access-8xq2j\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.824222 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.824169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.925379 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.925350 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b355a727-3563-42f4-825e-b18cfe2c9369-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.925571 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.925387 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.925571 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.925416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xq2j\" (UniqueName: \"kubernetes.io/projected/b355a727-3563-42f4-825e-b18cfe2c9369-kube-api-access-8xq2j\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.925571 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.925439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.925571 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.925480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b355a727-3563-42f4-825e-b18cfe2c9369-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.925905 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.925888 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b355a727-3563-42f4-825e-b18cfe2c9369-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.926201 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.926172 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.926201 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.926172 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.928215 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.928189 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b355a727-3563-42f4-825e-b18cfe2c9369-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:03.936577 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:03.936551 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xq2j\" (UniqueName: \"kubernetes.io/projected/b355a727-3563-42f4-825e-b18cfe2c9369-kube-api-access-8xq2j\") pod \"isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:04.024331 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.024304 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:04.178162 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.178137 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w"] Jul 01 19:31:04.180728 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:31:04.180691 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb355a727_3563_42f4_825e_b18cfe2c9369.slice/crio-f25e58d2cb33b84b75ad9b1a35ba6638f20694bd39dcde737cf211947bf59cb7 WatchSource:0}: Error finding container f25e58d2cb33b84b75ad9b1a35ba6638f20694bd39dcde737cf211947bf59cb7: Status 404 returned error can't find the container with id f25e58d2cb33b84b75ad9b1a35ba6638f20694bd39dcde737cf211947bf59cb7 Jul 01 19:31:04.189367 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.189341 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm"] Jul 01 19:31:04.193290 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.193270 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.195889 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.195864 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 01 19:31:04.196091 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.195890 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 01 19:31:04.204642 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.204575 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm"] Jul 01 19:31:04.329366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.329268 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.329366 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.329333 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.329565 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.329369 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.329565 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.329394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.329565 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.329477 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd5wb\" (UniqueName: \"kubernetes.io/projected/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-api-access-jd5wb\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.430347 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.430312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd5wb\" (UniqueName: \"kubernetes.io/projected/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-api-access-jd5wb\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.430512 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.430382 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.430512 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.430440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.430512 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.430471 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.430512 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.430499 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.431000 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.430976 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.431244 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.431226 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.431293 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.431253 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.432959 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.432930 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.439236 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.439211 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd5wb\" (UniqueName: \"kubernetes.io/projected/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-api-access-jd5wb\") pod \"isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.506175 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.506138 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:04.641561 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.641512 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm"] Jul 01 19:31:04.644333 ip-10-0-139-28 kubenswrapper[2570]: W0701 19:31:04.644307 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8f5e0a_b6c7_468a_a6f0_cb6abf69fc17.slice/crio-73ab2b829418dbc3483b642b81b0b8992f45a301ab7cfc617a923c8de32abd44 WatchSource:0}: Error finding container 73ab2b829418dbc3483b642b81b0b8992f45a301ab7cfc617a923c8de32abd44: Status 404 returned error can't find the container with id 73ab2b829418dbc3483b642b81b0b8992f45a301ab7cfc617a923c8de32abd44 Jul 01 19:31:04.823404 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.823360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerStarted","Data":"485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156"} Jul 01 19:31:04.823870 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.823436 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerStarted","Data":"f25e58d2cb33b84b75ad9b1a35ba6638f20694bd39dcde737cf211947bf59cb7"} Jul 01 19:31:04.825095 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.825069 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerStarted","Data":"0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97"} Jul 01 19:31:04.825209 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:04.825120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerStarted","Data":"73ab2b829418dbc3483b642b81b0b8992f45a301ab7cfc617a923c8de32abd44"} Jul 01 19:31:07.838153 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:07.838122 2570 generic.go:356] "Generic (PLEG): container finished" podID="b355a727-3563-42f4-825e-b18cfe2c9369" containerID="485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156" exitCode=0 Jul 01 19:31:07.838508 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:07.838193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerDied","Data":"485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156"} Jul 01 19:31:08.847657 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:08.846764 2570 generic.go:356] "Generic (PLEG): container finished" podID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerID="0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97" exitCode=0 Jul 01 19:31:08.847657 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:08.846872 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerDied","Data":"0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97"} Jul 01 19:31:23.943720 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:23.943654 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerStarted","Data":"acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b"} Jul 01 19:31:23.946096 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:23.946063 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerStarted","Data":"4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a"} Jul 01 19:31:25.961475 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:25.961446 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerStarted","Data":"062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02"} Jul 01 19:31:25.961887 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:25.961655 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:25.984606 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:25.984549 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podStartSLOduration=1.375490776 podStartE2EDuration="21.984530486s" totalImagesPullingTime="16.961117894s" totalInitContainerRuntime="3.647921816s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:31:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:31:25.983503738 +0000 UTC m=+598.096052338" watchObservedRunningTime="2026-07-01 19:31:25.984530486 +0000 UTC m=+598.097079085" Jul 01 19:31:26.970852 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.970817 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerStarted","Data":"844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193"} Jul 01 19:31:26.971295 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.970885 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:26.971295 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.970915 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:26.971295 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.971064 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:26.972371 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.972339 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:31:26.972484 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.972358 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:31:26.996159 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:26.996112 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podStartSLOduration=2.458204271 podStartE2EDuration="23.996096894s" totalImagesPullingTime="17.981601628s" totalInitContainerRuntime="3.556290995s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:31:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:31:26.992894562 +0000 UTC m=+599.105443161" watchObservedRunningTime="2026-07-01 19:31:26.996096894 +0000 UTC m=+599.108645493" Jul 01 19:31:27.975444 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:27.975407 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:31:27.975869 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:27.975408 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:31:28.341120 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:28.341089 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:31:28.341120 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:28.341106 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:31:32.979405 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:32.979377 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:31:32.979847 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:32.979824 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:31:32.979907 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:32.979866 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:31:32.980315 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:32.980291 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:31:42.980073 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:42.980020 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:31:42.980670 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:42.980239 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:31:52.980840 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:52.980749 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:31:52.981233 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:31:52.980756 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:32:02.979988 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:02.979945 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:32:02.980489 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:02.980407 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:32:12.979880 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:12.979838 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:32:12.981226 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:12.980184 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:32:22.980292 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:22.980249 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:32:22.980739 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:22.980249 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:32:32.980238 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:32.980193 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Jul 01 19:32:32.980663 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:32.980337 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Jul 01 19:32:42.980401 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:42.980370 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:32:42.981192 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:32:42.981177 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:33:13.898913 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:13.898877 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm"] Jul 01 19:33:13.899480 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:13.899092 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" containerID="cri-o://4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a" gracePeriod=30 Jul 01 19:33:13.899480 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:13.899136 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kube-rbac-proxy" containerID="cri-o://062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02" gracePeriod=30 Jul 01 19:33:13.964421 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:13.964389 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w"] Jul 01 19:33:13.964707 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:13.964683 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" containerID="cri-o://acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b" gracePeriod=30 Jul 01 19:33:13.964823 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:13.964796 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kube-rbac-proxy" containerID="cri-o://844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193" gracePeriod=30 Jul 01 19:33:14.408648 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:14.408587 2570 generic.go:356] "Generic (PLEG): container finished" podID="b355a727-3563-42f4-825e-b18cfe2c9369" containerID="844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193" exitCode=2 Jul 01 19:33:14.408822 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:14.408664 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerDied","Data":"844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193"} Jul 01 19:33:14.410613 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:14.410591 2570 generic.go:356] "Generic (PLEG): container finished" podID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerID="062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02" exitCode=2 Jul 01 19:33:14.410745 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:14.410659 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerDied","Data":"062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02"} Jul 01 19:33:17.976105 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:17.976062 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.20:8643/healthz\": dial tcp 10.134.0.20:8643: connect: connection refused" Jul 01 19:33:17.976493 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:17.976065 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused" Jul 01 19:33:18.950791 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.950767 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:33:18.980862 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.980834 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-rbac-proxy-sar-config\") pod \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " Jul 01 19:33:18.981309 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.980899 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd5wb\" (UniqueName: \"kubernetes.io/projected/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-api-access-jd5wb\") pod \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " Jul 01 19:33:18.981309 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.980945 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kserve-provision-location\") pod \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " Jul 01 19:33:18.981309 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.981011 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-cabundle-cert\") pod \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " Jul 01 19:33:18.981309 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.981038 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-proxy-tls\") pod \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\" (UID: \"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17\") " Jul 01 19:33:18.981504 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.981350 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" (UID: "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:33:18.981504 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.981424 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" (UID: "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:33:18.981504 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.981438 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" (UID: "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:33:18.985834 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.985773 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" (UID: "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:33:18.985933 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:18.985898 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-api-access-jd5wb" (OuterVolumeSpecName: "kube-api-access-jd5wb") pod "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" (UID: "9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17"). InnerVolumeSpecName "kube-api-access-jd5wb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:33:19.009010 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.008986 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:33:19.081857 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.081820 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-cabundle-cert\") pod \"b355a727-3563-42f4-825e-b18cfe2c9369\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " Jul 01 19:33:19.082033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.081875 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b355a727-3563-42f4-825e-b18cfe2c9369-kserve-provision-location\") pod \"b355a727-3563-42f4-825e-b18cfe2c9369\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " Jul 01 19:33:19.082033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.081900 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b355a727-3563-42f4-825e-b18cfe2c9369-proxy-tls\") pod \"b355a727-3563-42f4-825e-b18cfe2c9369\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " Jul 01 19:33:19.082033 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082010 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-kube-rbac-proxy-sar-config\") pod \"b355a727-3563-42f4-825e-b18cfe2c9369\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " Jul 01 19:33:19.082172 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082080 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xq2j\" (UniqueName: \"kubernetes.io/projected/b355a727-3563-42f4-825e-b18cfe2c9369-kube-api-access-8xq2j\") pod \"b355a727-3563-42f4-825e-b18cfe2c9369\" (UID: \"b355a727-3563-42f4-825e-b18cfe2c9369\") " Jul 01 19:33:19.082262 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082229 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b355a727-3563-42f4-825e-b18cfe2c9369-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b355a727-3563-42f4-825e-b18cfe2c9369" (UID: "b355a727-3563-42f4-825e-b18cfe2c9369"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 19:33:19.082326 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082267 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b355a727-3563-42f4-825e-b18cfe2c9369" (UID: "b355a727-3563-42f4-825e-b18cfe2c9369"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:33:19.082382 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082336 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b355a727-3563-42f4-825e-b18cfe2c9369" (UID: "b355a727-3563-42f4-825e-b18cfe2c9369"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:33:19.082436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082388 2570 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b355a727-3563-42f4-825e-b18cfe2c9369-kserve-provision-location\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.082436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082400 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd5wb\" (UniqueName: \"kubernetes.io/projected/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-api-access-jd5wb\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.082436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082411 2570 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kserve-provision-location\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.082436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082421 2570 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-cabundle-cert\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.082436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082430 2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-proxy-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.082436 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082437 2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.082679 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.082447 2570 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-cabundle-cert\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.083936 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.083917 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b355a727-3563-42f4-825e-b18cfe2c9369-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b355a727-3563-42f4-825e-b18cfe2c9369" (UID: "b355a727-3563-42f4-825e-b18cfe2c9369"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:33:19.084041 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.084019 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b355a727-3563-42f4-825e-b18cfe2c9369-kube-api-access-8xq2j" (OuterVolumeSpecName: "kube-api-access-8xq2j") pod "b355a727-3563-42f4-825e-b18cfe2c9369" (UID: "b355a727-3563-42f4-825e-b18cfe2c9369"). InnerVolumeSpecName "kube-api-access-8xq2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:33:19.183682 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.183608 2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b355a727-3563-42f4-825e-b18cfe2c9369-proxy-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.183682 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.183676 2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b355a727-3563-42f4-825e-b18cfe2c9369-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.183682 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.183689 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xq2j\" (UniqueName: \"kubernetes.io/projected/b355a727-3563-42f4-825e-b18cfe2c9369-kube-api-access-8xq2j\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 19:33:19.433822 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.433781 2570 generic.go:356] "Generic (PLEG): container finished" podID="b355a727-3563-42f4-825e-b18cfe2c9369" containerID="acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b" exitCode=0 Jul 01 19:33:19.434015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.433851 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" Jul 01 19:33:19.434015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.433855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerDied","Data":"acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b"} Jul 01 19:33:19.434015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.433917 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w" event={"ID":"b355a727-3563-42f4-825e-b18cfe2c9369","Type":"ContainerDied","Data":"f25e58d2cb33b84b75ad9b1a35ba6638f20694bd39dcde737cf211947bf59cb7"} Jul 01 19:33:19.434015 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.433945 2570 scope.go:117] "RemoveContainer" containerID="844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193" Jul 01 19:33:19.435856 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.435831 2570 generic.go:356] "Generic (PLEG): container finished" podID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerID="4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a" exitCode=0 Jul 01 19:33:19.435985 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.435891 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" Jul 01 19:33:19.435985 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.435906 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerDied","Data":"4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a"} Jul 01 19:33:19.435985 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.435962 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm" event={"ID":"9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17","Type":"ContainerDied","Data":"73ab2b829418dbc3483b642b81b0b8992f45a301ab7cfc617a923c8de32abd44"} Jul 01 19:33:19.442350 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.442324 2570 scope.go:117] "RemoveContainer" containerID="acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b" Jul 01 19:33:19.449994 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.449981 2570 scope.go:117] "RemoveContainer" containerID="485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156" Jul 01 19:33:19.458932 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.458913 2570 scope.go:117] "RemoveContainer" containerID="844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193" Jul 01 19:33:19.459241 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:33:19.459224 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193\": container with ID starting with 844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193 not found: ID does not exist" containerID="844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193" Jul 01 19:33:19.459306 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459250 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193"} err="failed to get container status \"844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193\": rpc error: code = NotFound desc = could not find container \"844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193\": container with ID starting with 844e9074e8c99b7f85a87361f39bba391c688ddbf911654018e5edf50d6b2193 not found: ID does not exist" Jul 01 19:33:19.459306 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459266 2570 scope.go:117] "RemoveContainer" containerID="acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b" Jul 01 19:33:19.459397 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459377 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w"] Jul 01 19:33:19.459518 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:33:19.459503 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b\": container with ID starting with acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b not found: ID does not exist" containerID="acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b" Jul 01 19:33:19.459558 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459521 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b"} err="failed to get container status \"acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b\": rpc error: code = NotFound desc = could not find container \"acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b\": container with ID starting with acce1eca99a1863f06b17efdfc1074cd5af399841cd06211000fad4b7bfb549b not found: ID does not exist" Jul 01 19:33:19.459558 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459545 2570 scope.go:117] "RemoveContainer" containerID="485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156" Jul 01 19:33:19.459826 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:33:19.459809 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156\": container with ID starting with 485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156 not found: ID does not exist" containerID="485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156" Jul 01 19:33:19.459884 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459830 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156"} err="failed to get container status \"485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156\": rpc error: code = NotFound desc = could not find container \"485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156\": container with ID starting with 485bc43aaa242f193267f47c54a2677e44a65e9dc7825366a7ecac90a4ddc156 not found: ID does not exist" Jul 01 19:33:19.459884 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.459844 2570 scope.go:117] "RemoveContainer" containerID="062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02" Jul 01 19:33:19.463369 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.463351 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6ccb86d5d8-8kf9w"] Jul 01 19:33:19.467384 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.467368 2570 scope.go:117] "RemoveContainer" containerID="4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a" Jul 01 19:33:19.474507 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.474480 2570 scope.go:117] "RemoveContainer" containerID="0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97" Jul 01 19:33:19.474846 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.474830 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm"] Jul 01 19:33:19.479166 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.479148 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b45d9b44-l4hwm"] Jul 01 19:33:19.481845 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.481829 2570 scope.go:117] "RemoveContainer" containerID="062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02" Jul 01 19:33:19.482115 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:33:19.482098 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02\": container with ID starting with 062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02 not found: ID does not exist" containerID="062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02" Jul 01 19:33:19.482162 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.482122 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02"} err="failed to get container status \"062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02\": rpc error: code = NotFound desc = could not find container \"062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02\": container with ID starting with 062897430b67ae24c5545efbe5154186f97263e1380865da1ea0d521c2864a02 not found: ID does not exist" Jul 01 19:33:19.482162 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.482150 2570 scope.go:117] "RemoveContainer" containerID="4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a" Jul 01 19:33:19.482361 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:33:19.482346 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a\": container with ID starting with 4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a not found: ID does not exist" containerID="4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a" Jul 01 19:33:19.482409 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.482364 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a"} err="failed to get container status \"4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a\": rpc error: code = NotFound desc = could not find container \"4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a\": container with ID starting with 4b941b48ecc71104415655492f576941d6035b1ce9bffcc3f104d6a6a535e72a not found: ID does not exist" Jul 01 19:33:19.482409 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.482376 2570 scope.go:117] "RemoveContainer" containerID="0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97" Jul 01 19:33:19.482580 ip-10-0-139-28 kubenswrapper[2570]: E0701 19:33:19.482564 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97\": container with ID starting with 0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97 not found: ID does not exist" containerID="0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97" Jul 01 19:33:19.482712 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:19.482584 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97"} err="failed to get container status \"0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97\": rpc error: code = NotFound desc = could not find container \"0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97\": container with ID starting with 0c5552d6b05269dd78316496d597661acecb2dab21816ba9df22b04326313c97 not found: ID does not exist" Jul 01 19:33:20.360209 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:20.360177 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" path="/var/lib/kubelet/pods/9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17/volumes" Jul 01 19:33:20.360693 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:33:20.360680 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" path="/var/lib/kubelet/pods/b355a727-3563-42f4-825e-b18cfe2c9369/volumes" Jul 01 19:36:28.368136 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:36:28.368051 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:36:28.368825 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:36:28.368803 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:41:28.397044 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:41:28.397013 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:41:28.398379 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:41:28.398357 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:46:28.423348 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:46:28.423317 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:46:28.426049 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:46:28.426025 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:51:28.455245 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:51:28.455214 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:51:28.457998 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:51:28.457967 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:56:28.478322 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:56:28.478294 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 19:56:28.481495 ip-10-0-139-28 kubenswrapper[2570]: I0701 19:56:28.481475 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 20:01:28.502767 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:01:28.502651 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 20:01:28.506660 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:01:28.506251 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 20:06:28.530110 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:06:28.530083 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 20:06:28.535116 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:06:28.535099 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 20:10:14.451471 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451439 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ktwlx/must-gather-nmncs"] Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451745 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="storage-initializer" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451757 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="storage-initializer" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451769 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451775 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451782 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kube-rbac-proxy" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451787 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kube-rbac-proxy" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451796 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kube-rbac-proxy" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451801 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kube-rbac-proxy" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451810 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451816 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451825 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="storage-initializer" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451832 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="storage-initializer" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451873 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kserve-container" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451884 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b355a727-3563-42f4-825e-b18cfe2c9369" containerName="kube-rbac-proxy" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451892 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kube-rbac-proxy" Jul 01 20:10:14.451986 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.451898 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8f5e0a-b6c7-468a-a6f0-cb6abf69fc17" containerName="kserve-container" Jul 01 20:10:14.454487 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.454466 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.457132 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.457092 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ktwlx\"/\"kube-root-ca.crt\"" Jul 01 20:10:14.457244 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.457106 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ktwlx\"/\"openshift-service-ca.crt\"" Jul 01 20:10:14.457244 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.457104 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ktwlx\"/\"default-dockercfg-85rkc\"" Jul 01 20:10:14.464185 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.464159 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ktwlx/must-gather-nmncs"] Jul 01 20:10:14.556434 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.556401 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-must-gather-output\") pod \"must-gather-nmncs\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.556617 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.556474 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfxm\" (UniqueName: \"kubernetes.io/projected/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-kube-api-access-4sfxm\") pod \"must-gather-nmncs\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.657468 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.657434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-must-gather-output\") pod \"must-gather-nmncs\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.657654 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.657501 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfxm\" (UniqueName: \"kubernetes.io/projected/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-kube-api-access-4sfxm\") pod \"must-gather-nmncs\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.657740 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.657723 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-must-gather-output\") pod \"must-gather-nmncs\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.666075 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.666054 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfxm\" (UniqueName: \"kubernetes.io/projected/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-kube-api-access-4sfxm\") pod \"must-gather-nmncs\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.764317 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.764231 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:14.892864 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.892779 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ktwlx/must-gather-nmncs"] Jul 01 20:10:14.895401 ip-10-0-139-28 kubenswrapper[2570]: W0701 20:10:14.895381 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod520a79cd_fbc9_4f1f_84e4_83fdc9f203f3.slice/crio-41c435f8d5fb560bd0c0988aa6ad1e1c75c1a7072f5f6df357d8c431829fd3eb WatchSource:0}: Error finding container 41c435f8d5fb560bd0c0988aa6ad1e1c75c1a7072f5f6df357d8c431829fd3eb: Status 404 returned error can't find the container with id 41c435f8d5fb560bd0c0988aa6ad1e1c75c1a7072f5f6df357d8c431829fd3eb Jul 01 20:10:14.897169 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:14.897150 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 20:10:15.795749 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:15.795702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ktwlx/must-gather-nmncs" event={"ID":"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3","Type":"ContainerStarted","Data":"41c435f8d5fb560bd0c0988aa6ad1e1c75c1a7072f5f6df357d8c431829fd3eb"} Jul 01 20:10:21.826400 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:21.826362 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ktwlx/must-gather-nmncs" event={"ID":"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3","Type":"ContainerStarted","Data":"d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e"} Jul 01 20:10:21.826400 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:21.826406 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ktwlx/must-gather-nmncs" event={"ID":"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3","Type":"ContainerStarted","Data":"b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0"} Jul 01 20:10:21.844467 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:21.844417 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ktwlx/must-gather-nmncs" podStartSLOduration=1.607726284 podStartE2EDuration="7.844401766s" totalImagesPullingTime="6.236675482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 20:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:10:21.842967285 +0000 UTC m=+2933.955515887" watchObservedRunningTime="2026-07-01 20:10:21.844401766 +0000 UTC m=+2933.956950391" Jul 01 20:10:41.911856 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:41.911821 2570 generic.go:356] "Generic (PLEG): container finished" podID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerID="b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0" exitCode=0 Jul 01 20:10:41.912254 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:41.911869 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ktwlx/must-gather-nmncs" event={"ID":"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3","Type":"ContainerDied","Data":"b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0"} Jul 01 20:10:41.912254 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:41.912141 2570 scope.go:117] "RemoveContainer" containerID="b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0" Jul 01 20:10:42.494328 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:42.494303 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-ktwlx_must-gather-nmncs_520a79cd-fbc9-4f1f-84e4-83fdc9f203f3/gather/0.log" Jul 01 20:10:45.835937 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:45.835906 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vnj75_a4761df2-8e35-4f4a-a6f7-20363c365ca4/global-pull-secret-syncer/0.log" Jul 01 20:10:45.923078 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:45.923049 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fhmrc_0998dc6a-abaf-4aaa-ac92-34ea5269afc3/konnectivity-agent/0.log" Jul 01 20:10:45.997517 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:45.997488 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-28.ec2.internal_e757ec6aacc35d99d6a9719bfafce01a/haproxy/0.log" Jul 01 20:10:48.019073 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.019034 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-ktwlx/must-gather-nmncs"] Jul 01 20:10:48.019604 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.019257 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-ktwlx/must-gather-nmncs" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="copy" containerID="cri-o://d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e" gracePeriod=2 Jul 01 20:10:48.023941 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.023915 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-ktwlx/must-gather-nmncs"] Jul 01 20:10:48.254252 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.254227 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-ktwlx_must-gather-nmncs_520a79cd-fbc9-4f1f-84e4-83fdc9f203f3/copy/0.log" Jul 01 20:10:48.255660 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.255644 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:48.258146 ip-10-0-139-28 kubenswrapper[2570]: E0701 20:10:48.258117 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nmncs\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-ktwlx\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:48.349219 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.349144 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-must-gather-output\") pod \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " Jul 01 20:10:48.349219 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.349179 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sfxm\" (UniqueName: \"kubernetes.io/projected/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-kube-api-access-4sfxm\") pod \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\" (UID: \"520a79cd-fbc9-4f1f-84e4-83fdc9f203f3\") " Jul 01 20:10:48.350838 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.350809 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" (UID: "520a79cd-fbc9-4f1f-84e4-83fdc9f203f3"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 20:10:48.351355 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.351329 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-kube-api-access-4sfxm" (OuterVolumeSpecName: "kube-api-access-4sfxm") pod "520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" (UID: "520a79cd-fbc9-4f1f-84e4-83fdc9f203f3"). InnerVolumeSpecName "kube-api-access-4sfxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 20:10:48.359287 ip-10-0-139-28 kubenswrapper[2570]: E0701 20:10:48.359257 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nmncs\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-ktwlx\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:48.360231 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.360212 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" path="/var/lib/kubelet/pods/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3/volumes" Jul 01 20:10:48.449744 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.449704 2570 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-must-gather-output\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 20:10:48.449744 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.449736 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sfxm\" (UniqueName: \"kubernetes.io/projected/520a79cd-fbc9-4f1f-84e4-83fdc9f203f3-kube-api-access-4sfxm\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Jul 01 20:10:48.939405 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.939377 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-ktwlx_must-gather-nmncs_520a79cd-fbc9-4f1f-84e4-83fdc9f203f3/copy/0.log" Jul 01 20:10:48.939763 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.939736 2570 generic.go:356] "Generic (PLEG): container finished" podID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerID="d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e" exitCode=143 Jul 01 20:10:48.939876 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.939816 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ktwlx/must-gather-nmncs" Jul 01 20:10:48.939876 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.939831 2570 scope.go:117] "RemoveContainer" containerID="d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e" Jul 01 20:10:48.947302 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.947285 2570 scope.go:117] "RemoveContainer" containerID="b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0" Jul 01 20:10:48.959206 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.959190 2570 scope.go:117] "RemoveContainer" containerID="d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e" Jul 01 20:10:48.959454 ip-10-0-139-28 kubenswrapper[2570]: E0701 20:10:48.959434 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e\": container with ID starting with d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e not found: ID does not exist" containerID="d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e" Jul 01 20:10:48.959507 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.959462 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e"} err="failed to get container status \"d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e\": rpc error: code = NotFound desc = could not find container \"d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e\": container with ID starting with d3a12b3908edfd124f8be842c088ff9d28a8ef4d186507490fa36734088fa60e not found: ID does not exist" Jul 01 20:10:48.959507 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.959479 2570 scope.go:117] "RemoveContainer" containerID="b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0" Jul 01 20:10:48.959706 ip-10-0-139-28 kubenswrapper[2570]: E0701 20:10:48.959689 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0\": container with ID starting with b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0 not found: ID does not exist" containerID="b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0" Jul 01 20:10:48.959764 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:48.959717 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0"} err="failed to get container status \"b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0\": rpc error: code = NotFound desc = could not find container \"b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0\": container with ID starting with b4fcbfc2183c9db0c94dd3739711702773d382db7247305edbe3d997fee2aaf0 not found: ID does not exist" Jul 01 20:10:50.383186 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.383106 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-lxr4p_7531648a-08e8-454c-983c-3da6fab2b2cc/monitoring-plugin/0.log" Jul 01 20:10:50.419936 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.419909 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88f4d_67f25080-984f-4295-85e1-68a9fae8fbb7/node-exporter/0.log" Jul 01 20:10:50.441872 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.441851 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88f4d_67f25080-984f-4295-85e1-68a9fae8fbb7/kube-rbac-proxy/0.log" Jul 01 20:10:50.466558 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.466536 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88f4d_67f25080-984f-4295-85e1-68a9fae8fbb7/init-textfile/0.log" Jul 01 20:10:50.671508 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.671443 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-nsnbj_dd537fa1-b864-433e-8546-be82b2614c99/kube-rbac-proxy-main/0.log" Jul 01 20:10:50.694263 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.694241 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-nsnbj_dd537fa1-b864-433e-8546-be82b2614c99/kube-rbac-proxy-self/0.log" Jul 01 20:10:50.717752 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.717726 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-nsnbj_dd537fa1-b864-433e-8546-be82b2614c99/openshift-state-metrics/0.log" Jul 01 20:10:50.763835 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.763808 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/prometheus/0.log" Jul 01 20:10:50.784952 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.784926 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/config-reloader/0.log" Jul 01 20:10:50.813543 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.813520 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/thanos-sidecar/0.log" Jul 01 20:10:50.840712 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.840690 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/kube-rbac-proxy-web/0.log" Jul 01 20:10:50.866959 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.866940 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/kube-rbac-proxy/0.log" Jul 01 20:10:50.894478 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.894456 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/kube-rbac-proxy-thanos/0.log" Jul 01 20:10:50.921366 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:50.921347 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f7a05c-2620-48c5-bbaa-f0d112b17636/init-config-reloader/0.log" Jul 01 20:10:51.005500 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.005429 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-tpcjr_c7224345-e559-47d4-bd8f-bd318d0ad9cd/prometheus-operator-admission-webhook/0.log" Jul 01 20:10:51.115475 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.115441 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86dd7bdd8b-n6xl9_578f85e0-94fc-471d-b722-9b8458e824df/thanos-query/0.log" Jul 01 20:10:51.146507 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.146481 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86dd7bdd8b-n6xl9_578f85e0-94fc-471d-b722-9b8458e824df/kube-rbac-proxy-web/0.log" Jul 01 20:10:51.168995 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.168970 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86dd7bdd8b-n6xl9_578f85e0-94fc-471d-b722-9b8458e824df/kube-rbac-proxy/0.log" Jul 01 20:10:51.189326 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.189305 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86dd7bdd8b-n6xl9_578f85e0-94fc-471d-b722-9b8458e824df/prom-label-proxy/0.log" Jul 01 20:10:51.215983 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.215963 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86dd7bdd8b-n6xl9_578f85e0-94fc-471d-b722-9b8458e824df/kube-rbac-proxy-rules/0.log" Jul 01 20:10:51.238267 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:51.238249 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86dd7bdd8b-n6xl9_578f85e0-94fc-471d-b722-9b8458e824df/kube-rbac-proxy-metrics/0.log" Jul 01 20:10:54.139456 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139420 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg"] Jul 01 20:10:54.139960 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139762 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="gather" Jul 01 20:10:54.139960 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139775 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="gather" Jul 01 20:10:54.139960 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139782 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="copy" Jul 01 20:10:54.139960 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139788 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="copy" Jul 01 20:10:54.139960 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139838 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="copy" Jul 01 20:10:54.139960 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.139847 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="520a79cd-fbc9-4f1f-84e4-83fdc9f203f3" containerName="gather" Jul 01 20:10:54.145059 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.145035 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.147476 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.147443 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2sjbp\"/\"kube-root-ca.crt\"" Jul 01 20:10:54.148580 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.148558 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2sjbp\"/\"default-dockercfg-m98cx\"" Jul 01 20:10:54.148724 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.148689 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2sjbp\"/\"openshift-service-ca.crt\"" Jul 01 20:10:54.153707 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.153684 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg"] Jul 01 20:10:54.192698 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.192665 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-sys\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.192846 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.192738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-podres\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.192846 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.192797 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-proc\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.192846 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.192828 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mh8d\" (UniqueName: \"kubernetes.io/projected/cf87db1d-0e34-476f-8809-a1f0f5b7e267-kube-api-access-7mh8d\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.192974 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.192852 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-lib-modules\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.293945 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.293900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-podres\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.293945 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.293957 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-proc\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294187 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.293982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mh8d\" (UniqueName: \"kubernetes.io/projected/cf87db1d-0e34-476f-8809-a1f0f5b7e267-kube-api-access-7mh8d\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294187 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.294050 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-proc\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294187 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.294120 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-podres\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294187 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.294139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-lib-modules\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294187 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.294186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-sys\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294366 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.294288 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-sys\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.294366 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.294300 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf87db1d-0e34-476f-8809-a1f0f5b7e267-lib-modules\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.305716 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.305691 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mh8d\" (UniqueName: \"kubernetes.io/projected/cf87db1d-0e34-476f-8809-a1f0f5b7e267-kube-api-access-7mh8d\") pod \"perf-node-gather-daemonset-bc8wg\" (UID: \"cf87db1d-0e34-476f-8809-a1f0f5b7e267\") " pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.455716 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.455608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.578378 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.578287 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg"] Jul 01 20:10:54.580513 ip-10-0-139-28 kubenswrapper[2570]: W0701 20:10:54.580479 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcf87db1d_0e34_476f_8809_a1f0f5b7e267.slice/crio-ca809bcafc45b35fd4585e14fa8f0d8c089f507084c11a065592e07aec2e369d WatchSource:0}: Error finding container ca809bcafc45b35fd4585e14fa8f0d8c089f507084c11a065592e07aec2e369d: Status 404 returned error can't find the container with id ca809bcafc45b35fd4585e14fa8f0d8c089f507084c11a065592e07aec2e369d Jul 01 20:10:54.950706 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.950679 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g9r8t_d415df21-c25d-439a-8147-9d82e9c13079/dns/0.log" Jul 01 20:10:54.965843 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.965813 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" event={"ID":"cf87db1d-0e34-476f-8809-a1f0f5b7e267","Type":"ContainerStarted","Data":"5b3c3b4a91861995180f212f4321889464330c9bb8ebd9dc42c36a3e29fe955d"} Jul 01 20:10:54.965992 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.965870 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" event={"ID":"cf87db1d-0e34-476f-8809-a1f0f5b7e267","Type":"ContainerStarted","Data":"ca809bcafc45b35fd4585e14fa8f0d8c089f507084c11a065592e07aec2e369d"} Jul 01 20:10:54.965992 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.965899 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:10:54.971276 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.971253 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g9r8t_d415df21-c25d-439a-8147-9d82e9c13079/kube-rbac-proxy/0.log" Jul 01 20:10:54.983700 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:54.983656 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" podStartSLOduration=0.983642239 podStartE2EDuration="983.642239ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 20:10:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 20:10:54.983140635 +0000 UTC m=+2967.095689236" watchObservedRunningTime="2026-07-01 20:10:54.983642239 +0000 UTC m=+2967.096190835" Jul 01 20:10:55.092368 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:55.092346 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xqnc9_749f1789-dc86-45ff-b45a-e39051e99144/dns-node-resolver/0.log" Jul 01 20:10:55.567469 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:55.567414 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54b569d4b9-trr5h_332c5816-d00f-46a0-89f7-ab17cff8db95/registry/0.log" Jul 01 20:10:55.588418 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:55.588396 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d5cbr_0f3bf7f8-6c2b-4600-a959-65d0bbd34195/node-ca/0.log" Jul 01 20:10:56.835638 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:56.835600 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbh8p_909f7802-3f30-4305-888f-7d688cd0e003/serve-healthcheck-canary/0.log" Jul 01 20:10:57.420470 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:57.420446 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzdzg_8ebd2222-c969-4a62-a631-4a059587a11e/kube-rbac-proxy/0.log" Jul 01 20:10:57.444464 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:57.444436 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzdzg_8ebd2222-c969-4a62-a631-4a059587a11e/exporter/0.log" Jul 01 20:10:57.469005 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:57.468978 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzdzg_8ebd2222-c969-4a62-a631-4a059587a11e/extractor/0.log" Jul 01 20:10:59.749575 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:59.749549 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-blw8w_c34d2cf9-67bf-4735-893b-14c0d08cba70/s3-init/0.log" Jul 01 20:10:59.765495 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:10:59.765469 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-blw8w_c34d2cf9-67bf-4735-893b-14c0d08cba70/download-hf-model/0.log" Jul 01 20:11:00.979503 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:00.979476 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2sjbp/perf-node-gather-daemonset-bc8wg" Jul 01 20:11:03.689196 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:03.689165 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-nrds2_d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f/migrator/0.log" Jul 01 20:11:03.717703 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:03.717676 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-nrds2_d67cc2f7-1edb-4cb6-a20f-3139c2a71c1f/graceful-termination/0.log" Jul 01 20:11:05.316141 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.316116 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/kube-multus-additional-cni-plugins/0.log" Jul 01 20:11:05.339244 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.339218 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/egress-router-binary-copy/0.log" Jul 01 20:11:05.360799 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.360773 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/cni-plugins/0.log" Jul 01 20:11:05.381491 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.381461 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/bond-cni-plugin/0.log" Jul 01 20:11:05.400906 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.400868 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/routeoverride-cni/0.log" Jul 01 20:11:05.420577 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.420551 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/whereabouts-cni-bincopy/0.log" Jul 01 20:11:05.440346 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.440322 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-l8zkw_7e8570e9-1bef-478c-9ca8-f55188dbc578/whereabouts-cni/0.log" Jul 01 20:11:05.496599 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.496572 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cj5j8_6d4e0861-3883-435e-a7fc-a7e193bb410c/kube-multus/0.log" Jul 01 20:11:05.520096 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.520071 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d5xdw_622a52e8-4365-440e-8a21-3c13ff32a70e/network-metrics-daemon/0.log" Jul 01 20:11:05.540732 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:05.540709 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d5xdw_622a52e8-4365-440e-8a21-3c13ff32a70e/kube-rbac-proxy/0.log" Jul 01 20:11:06.354759 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.354727 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-controller/0.log" Jul 01 20:11:06.374434 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.374402 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/1.log" Jul 01 20:11:06.374608 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.374476 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovn-acl-logging/0.log" Jul 01 20:11:06.390166 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.390139 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/kube-rbac-proxy-node/0.log" Jul 01 20:11:06.409974 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.409945 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 20:11:06.429427 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.429401 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/northd/0.log" Jul 01 20:11:06.450447 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.450424 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/nbdb/0.log" Jul 01 20:11:06.470891 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.470869 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/sbdb/0.log" Jul 01 20:11:06.508231 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:06.508200 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-27vrc_7715c025-a35d-4b2b-9ddf-460287a47e52/ovnkube-controller/0.log" Jul 01 20:11:07.971548 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:07.971519 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-j4rjt_be8a31ef-4a82-4e5b-9f13-934033eb47a3/network-check-target-container/0.log" Jul 01 20:11:08.912235 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:08.912208 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-pzrkd_11338b07-2eb1-432c-be08-260566adfa05/iptables-alerter/0.log" Jul 01 20:11:09.540645 ip-10-0-139-28 kubenswrapper[2570]: I0701 20:11:09.540580 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-mplxs_0ef997e1-e939-43d0-aaa5-dee6ebbb2aba/tuned/0.log"