Jul 07 11:38:03.256756 ip-10-0-138-138 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 11:38:03.256771 ip-10-0-138-138 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 11:38:03.256835 ip-10-0-138-138 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 11:38:03.257137 ip-10-0-138-138 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 11:38:13.305840 ip-10-0-138-138 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 11:38:13.305868 ip-10-0-138-138 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0b42b330657f4e0fb3cebc0873385155 -- Jul 07 11:40:11.047457 ip-10-0-138-138 systemd[1]: Starting Kubernetes Kubelet... Jul 07 11:40:11.515673 ip-10-0-138-138 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:40:11.515673 ip-10-0-138-138 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 11:40:11.515673 ip-10-0-138-138 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:40:11.515673 ip-10-0-138-138 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 11:40:11.515673 ip-10-0-138-138 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:40:11.516321 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.515700 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 11:40:11.523430 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523407 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:40:11.523430 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523425 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:40:11.523430 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523430 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:40:11.523430 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523433 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:40:11.523430 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523436 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:40:11.523430 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523439 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523443 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523446 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523449 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523452 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523455 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523458 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523461 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523463 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523466 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523492 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523496 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523500 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523503 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523506 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523509 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523512 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523515 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523518 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523521 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:40:11.523654 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523525 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523529 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523532 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523535 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523537 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523541 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523544 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523547 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523549 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523552 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523555 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523558 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523560 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523564 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523567 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523570 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523572 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523575 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523578 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523580 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:40:11.524148 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523585 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523589 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523592 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523595 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523598 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523601 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523604 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523606 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523609 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523612 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523615 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523618 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523620 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523623 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523626 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523629 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523632 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523635 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523638 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:40:11.524732 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523641 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523644 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523646 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523649 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523652 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523655 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523659 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523663 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523666 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523669 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523672 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523675 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523678 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523680 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523683 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523686 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523689 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523692 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523694 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523698 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:40:11.525206 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523700 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523703 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523706 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523710 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523714 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523717 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523720 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523723 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523726 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523729 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523732 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523734 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523737 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523740 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.523743 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524502 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524509 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524512 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524515 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:40:11.525723 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524519 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524522 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524525 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524528 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524531 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524534 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524537 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524539 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524542 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524545 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524548 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524551 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524556 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524560 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524564 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524569 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524573 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524576 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524579 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:40:11.526196 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524583 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524586 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524589 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524592 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524595 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524598 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524601 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524604 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524607 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524610 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524613 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524615 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524618 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524622 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524624 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524627 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524630 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524633 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524635 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524638 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:40:11.526702 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524641 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524643 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524646 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524648 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524651 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524655 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524657 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524660 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524663 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524666 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524668 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524671 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524674 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524677 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524679 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524682 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524684 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524687 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524690 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524692 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:40:11.527249 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524695 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524698 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524700 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524703 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524706 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524709 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524711 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524714 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524717 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524720 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524723 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524725 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524728 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524731 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524733 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524736 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524739 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524742 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524744 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524747 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:40:11.527773 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524750 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524753 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524759 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524762 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524765 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524769 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524772 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524774 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524777 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524780 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524783 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524785 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524788 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524791 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524794 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.524796 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525215 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525221 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525226 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:40:11.528294 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525230 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525233 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525236 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525238 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525241 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525244 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525247 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525249 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525252 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525255 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525258 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525260 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525263 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525284 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525289 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525293 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525296 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525299 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525303 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:40:11.528806 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525305 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525308 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525311 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525314 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525317 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525320 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525323 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525326 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525329 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525332 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525334 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525338 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525341 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525343 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525346 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525349 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525352 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525355 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525357 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525360 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:40:11.529303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525362 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525365 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525368 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525371 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525373 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525376 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525379 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525383 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525386 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525389 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525392 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525395 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525398 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525402 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525405 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525408 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525410 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525413 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525416 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525418 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:40:11.529817 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525421 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525424 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525426 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525429 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525432 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525434 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525437 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525439 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525445 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525448 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525450 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525453 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525456 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525459 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525461 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525464 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525466 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525469 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525472 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:40:11.530342 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525475 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525478 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525481 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525489 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525492 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525495 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525497 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525500 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525503 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525505 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525508 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525511 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525514 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525516 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525519 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525522 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525526 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.525529 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525594 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525601 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525608 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 11:40:11.530832 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525617 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525622 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525626 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525630 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525634 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525638 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525641 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525645 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525648 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525652 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525655 2580 flags.go:64] FLAG: --cgroup-root="" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525658 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525662 2580 flags.go:64] FLAG: --client-ca-file="" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525666 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525670 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525675 2580 flags.go:64] FLAG: --cluster-domain="" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525678 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525682 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525685 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525688 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525692 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525695 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525699 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 11:40:11.531408 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525703 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525706 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525709 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525712 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525715 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525718 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525723 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525726 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525729 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525734 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525737 2580 flags.go:64] FLAG: --enable-server="true" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525740 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525744 2580 flags.go:64] FLAG: --event-burst="100" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525748 2580 flags.go:64] FLAG: --event-qps="50" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525751 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525754 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525758 2580 flags.go:64] FLAG: --eviction-hard="" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525761 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525765 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525768 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525771 2580 flags.go:64] FLAG: --eviction-soft="" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525774 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525777 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525781 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525785 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 11:40:11.531988 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525788 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525791 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525794 2580 flags.go:64] FLAG: --feature-gates="" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525797 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525801 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525804 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525807 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525810 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525813 2580 flags.go:64] FLAG: --help="false" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525816 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525820 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525823 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525826 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525829 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525833 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525835 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525840 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525843 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525846 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525849 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525852 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525855 2580 flags.go:64] FLAG: --kube-reserved="" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525858 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525861 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 11:40:11.532807 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525864 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525867 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525870 2580 flags.go:64] FLAG: --lock-file="" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525873 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525876 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525879 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525885 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525889 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525892 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525895 2580 flags.go:64] FLAG: --logging-format="text" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525898 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525901 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525904 2580 flags.go:64] FLAG: --manifest-url="" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525907 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525912 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525915 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525919 2580 flags.go:64] FLAG: --max-pods="110" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525922 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525926 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525929 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525932 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525935 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525938 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525941 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525949 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 11:40:11.533911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525953 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525956 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525959 2580 flags.go:64] FLAG: --pod-cidr="" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525962 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525967 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525970 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525974 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525977 2580 flags.go:64] FLAG: --port="10250" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525980 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525983 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a14b44fd56f6d6e3" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525986 2580 flags.go:64] FLAG: --qos-reserved="" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525989 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525992 2580 flags.go:64] FLAG: --register-node="true" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525995 2580 flags.go:64] FLAG: --register-with-taints="" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.525999 2580 flags.go:64] FLAG: --registry-burst="10" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526002 2580 flags.go:64] FLAG: --registry-qps="5" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526005 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526008 2580 flags.go:64] FLAG: --reserved-memory="" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526012 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526015 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526018 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526021 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526024 2580 flags.go:64] FLAG: --runonce="false" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526027 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526030 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 11:40:11.535055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526033 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526036 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526039 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526042 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526045 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526048 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526052 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526055 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526058 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526061 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526065 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526068 2580 flags.go:64] FLAG: --system-cgroups="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526071 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526076 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526079 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526082 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526086 2580 flags.go:64] FLAG: --tls-min-version="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526089 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526092 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526095 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526098 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526101 2580 flags.go:64] FLAG: --v="2" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526105 2580 flags.go:64] FLAG: --version="false" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526110 2580 flags.go:64] FLAG: --vmodule="" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526114 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 11:40:11.536219 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.526117 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526234 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526239 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526243 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526246 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526248 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526252 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526254 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526257 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526260 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526263 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526280 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526283 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526288 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526291 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526296 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526299 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526301 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526304 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526307 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:40:11.537204 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526310 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526312 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526315 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526318 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526321 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526324 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526326 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526329 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526332 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526335 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526338 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526341 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526344 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526347 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526349 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526352 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526355 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526357 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526360 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526363 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:40:11.538044 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526366 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526368 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526371 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526374 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526376 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526381 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526383 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526387 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526390 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526393 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526396 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526399 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526402 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526404 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526407 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526410 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526413 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526415 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526418 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526421 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:40:11.538976 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526424 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526427 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526430 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526434 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526438 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526442 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526445 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526448 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526452 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526455 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526459 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526462 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526464 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526467 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526470 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526472 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526475 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526480 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526483 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:40:11.539689 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526487 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526490 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526493 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526496 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526498 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526501 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526504 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526507 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526510 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526513 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526515 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526518 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526521 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526524 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526527 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526529 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526532 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526535 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526538 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:40:11.540252 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526542 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.526545 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.527149 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.535119 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.535142 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.535166 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.535176 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535293 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535303 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535311 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535318 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535323 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535328 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535333 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535338 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:40:11.540782 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535343 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535348 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535355 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535361 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535366 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535371 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535376 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535381 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535386 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535391 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535395 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535400 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535404 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535409 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535413 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535417 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535422 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535426 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535431 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:40:11.541199 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535435 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535440 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535454 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535459 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535464 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535469 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535474 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535478 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535483 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535488 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535492 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535497 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535501 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535506 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535510 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535515 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535520 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535524 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535529 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535534 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:40:11.541730 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535538 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535543 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535547 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535552 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535557 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535561 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535567 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535572 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535576 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535581 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535585 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535590 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535594 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535599 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535607 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535621 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535627 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535633 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535637 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:40:11.542333 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535642 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535647 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535651 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535656 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535661 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535665 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535670 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535675 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535679 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535684 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535688 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535693 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535697 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535702 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535706 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535711 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535716 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535720 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535724 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535729 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:40:11.542856 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535735 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535739 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535744 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535748 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535753 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535758 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535762 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535767 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535771 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535784 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535790 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535794 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.535798 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.535807 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536041 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:40:11.543452 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536050 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536055 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536059 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536066 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536072 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536077 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536081 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536085 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536090 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536094 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536099 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536104 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536108 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536113 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536117 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536122 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536127 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536131 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536136 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:40:11.544140 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536140 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536145 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536150 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536154 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536161 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536167 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536172 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536178 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536191 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536196 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536201 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536206 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536211 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536215 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536220 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536225 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536229 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536234 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536239 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536244 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:40:11.545128 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536249 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536253 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536258 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536263 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536284 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536289 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536294 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536299 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536304 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536308 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536312 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536317 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536321 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536326 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536330 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536334 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536339 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536343 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536348 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:40:11.546027 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536352 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536356 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536382 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536388 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536392 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536397 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536401 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536406 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536411 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536416 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536420 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536425 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536429 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536433 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536438 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536442 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536447 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536451 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536456 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536460 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:40:11.546829 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536464 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536469 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536473 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536477 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536482 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536486 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536491 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536495 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536499 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536504 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536509 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536513 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536518 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536525 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536530 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536547 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536552 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536556 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536561 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:40:11.547440 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:11.536565 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.536573 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.536760 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.539584 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.541158 2580 server.go:1019] "Starting client certificate rotation" Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.541304 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.541349 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.544096 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:40:11.547957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.546592 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:40:11.560883 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.560860 2580 log.go:25] "Validated CRI v1 runtime API" Jul 07 11:40:11.563008 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.562991 2580 log.go:25] "Validated CRI v1 image API" Jul 07 11:40:11.563766 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.563749 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 11:40:11.567107 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.567086 2580 fs.go:135] Filesystem UUIDs: map[3756a97a-513d-456a-b03c-b61b267e8ddd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d1a03354-d7f4-431c-8d68-88e3a2609cf9:/dev/nvme0n1p4] Jul 07 11:40:11.567170 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.567106 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 11:40:11.571613 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.571594 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:40:11.572253 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.572131 2580 manager.go:216] Machine: {Timestamp:2026-07-07 11:40:11.570857213 +0000 UTC m=+0.414581307 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096422 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec288e097fa1ac8dd3115aba43059038 SystemUUID:ec288e09-7fa1-ac8d-d311-5aba43059038 BootID:0b42b330-657f-4e0f-b3ce-bc0873385155 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:84:93:27:65:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:84:93:27:65:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:69:b2:f4:2e:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 11:40:11.572253 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.572250 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 11:40:11.572371 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.572358 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 11:40:11.575416 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.575384 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 11:40:11.575554 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.575420 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-138.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 11:40:11.576184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.576174 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 11:40:11.576215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.576187 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 11:40:11.576215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.576195 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:40:11.576849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.576839 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:40:11.578126 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.578117 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 11:40:11.581284 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581272 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:40:11.581397 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581387 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 11:40:11.581435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581419 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 07 11:40:11.581469 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581438 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 11:40:11.581469 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581453 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 11:40:11.581469 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581462 2580 kubelet.go:398] "Adding apiserver pod source" Jul 07 11:40:11.581469 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581471 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 11:40:11.581899 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581882 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:40:11.581949 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.581913 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:40:11.584681 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.584662 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 11:40:11.586529 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.586515 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 11:40:11.586582 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.586547 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 11:40:11.587596 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587583 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587601 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587608 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587614 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587623 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587629 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587638 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587644 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587651 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587656 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587664 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 11:40:11.587676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.587670 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 11:40:11.588414 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.588402 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 11:40:11.588414 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.588414 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 11:40:11.591840 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.591824 2580 server.go:1268] "Started kubelet" Jul 07 11:40:11.591945 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.591890 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 11:40:11.592051 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.591915 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 11:40:11.592108 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.592077 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 11:40:11.592777 ip-10-0-138-138 systemd[1]: Started Kubernetes Kubelet. Jul 07 11:40:11.592909 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.592800 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 11:40:11.592909 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.592845 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-138.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 11:40:11.592909 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.592837 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 11:40:11.593588 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.593562 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 11:40:11.594799 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.594773 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 07 11:40:11.598304 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.598278 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 11:40:11.600188 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.600169 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qn4jz" Jul 07 11:40:11.602004 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.601031 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-138.ec2.internal.18bfff0da8b064fc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-138.ec2.internal,UID:ip-10-0-138-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-138.ec2.internal,},FirstTimestamp:2026-07-07 11:40:11.591795964 +0000 UTC m=+0.435520058,LastTimestamp:2026-07-07 11:40:11.591795964 +0000 UTC m=+0.435520058,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-138.ec2.internal,}" Jul 07 11:40:11.602229 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.602212 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 11:40:11.602229 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.602226 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 11:40:11.602389 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.602297 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:11.602876 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.602847 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 11:40:11.603008 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.602994 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 11:40:11.603140 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603069 2580 factory.go:55] Registering systemd factory Jul 07 11:40:11.603227 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603149 2580 factory.go:223] Registration of the systemd container factory successfully Jul 07 11:40:11.603278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603263 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 07 11:40:11.603278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603275 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 07 11:40:11.603406 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603381 2580 factory.go:153] Registering CRI-O factory Jul 07 11:40:11.603477 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603410 2580 factory.go:223] Registration of the crio container factory successfully Jul 07 11:40:11.603477 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603444 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 11:40:11.603477 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603475 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 11:40:11.603638 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603499 2580 factory.go:103] Registering Raw factory Jul 07 11:40:11.603638 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.603512 2580 manager.go:1196] Started watching for new ooms in manager Jul 07 11:40:11.604406 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.604391 2580 manager.go:349] Starting recovery of all containers Jul 07 11:40:11.608131 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.608103 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 11:40:11.608253 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.608229 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 11:40:11.611714 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.611692 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qn4jz" Jul 07 11:40:11.615698 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.615683 2580 manager.go:354] Recovery completed Jul 07 11:40:11.620036 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.620023 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.621939 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.621923 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.622008 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.621949 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.622008 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.621959 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.622620 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.622597 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 11:40:11.622620 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.622614 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 11:40:11.622751 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.622615 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.622751 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.622633 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:40:11.623521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.623505 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.623599 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.623534 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.623599 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.623549 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.624081 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.624067 2580 policy_none.go:49] "None policy: Start" Jul 07 11:40:11.624121 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.624084 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 11:40:11.624121 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.624095 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 11:40:11.624528 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.624446 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-138.ec2.internal.18bfff0daa7c4e61 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-138.ec2.internal,UID:ip-10-0-138-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-138.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-138.ec2.internal,},FirstTimestamp:2026-07-07 11:40:11.621936737 +0000 UTC m=+0.465660831,LastTimestamp:2026-07-07 11:40:11.621936737 +0000 UTC m=+0.465660831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-138.ec2.internal,}" Jul 07 11:40:11.625530 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.625516 2580 policy_none.go:47] "Start" Jul 07 11:40:11.667895 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.667875 2580 manager.go:337] "Starting Device Plugin manager" Jul 07 11:40:11.668028 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.667916 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 11:40:11.668028 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.667930 2580 server.go:87] "Starting device plugin registration server" Jul 07 11:40:11.668214 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.668199 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 11:40:11.668307 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.668216 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 11:40:11.668361 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.668336 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 11:40:11.668631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.668418 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 11:40:11.668631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.668432 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 11:40:11.668947 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.668924 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 11:40:11.669036 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.669003 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:11.773000 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.772948 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.774192 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.773905 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.774192 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.773933 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.774192 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.773943 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.774192 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.773970 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.781386 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.781371 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.781441 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.781393 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-138.ec2.internal\": node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:11.795717 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.795693 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:11.827254 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.827232 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 11:40:11.828533 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.828516 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 11:40:11.828630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.828552 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 11:40:11.828630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.828571 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 11:40:11.828630 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.828602 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 11:40:11.831406 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.831389 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:40:11.896311 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.896274 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:11.928998 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.928966 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal"] Jul 07 11:40:11.929057 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.929040 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.930496 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.930481 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.930614 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.930510 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.930614 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.930522 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.931830 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.931817 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.931943 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.931931 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.931980 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.931960 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.932629 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.932613 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.932629 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.932625 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.932706 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.932643 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.932706 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.932643 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.932706 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.932656 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.932706 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.932663 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.933866 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.933853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.933911 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.933879 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:40:11.934545 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.934525 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:40:11.934628 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.934554 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:40:11.934628 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:11.934565 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:40:11.958109 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.958082 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-138.ec2.internal\" not found" node="ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.962799 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.962782 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-138.ec2.internal\" not found" node="ip-10-0-138-138.ec2.internal" Jul 07 11:40:11.996749 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:11.996728 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.004667 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.004647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28d643209d9562fc33aacf73fb6e4fab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal\" (UID: \"28d643209d9562fc33aacf73fb6e4fab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.004791 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.004672 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28d643209d9562fc33aacf73fb6e4fab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal\" (UID: \"28d643209d9562fc33aacf73fb6e4fab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.004791 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.004690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea773fa8591c47f0caeb4b4d0061f7eb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-138.ec2.internal\" (UID: \"ea773fa8591c47f0caeb4b4d0061f7eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.097792 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.097733 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.105700 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.105675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28d643209d9562fc33aacf73fb6e4fab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal\" (UID: \"28d643209d9562fc33aacf73fb6e4fab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.105762 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.105737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28d643209d9562fc33aacf73fb6e4fab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal\" (UID: \"28d643209d9562fc33aacf73fb6e4fab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.105804 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.105773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28d643209d9562fc33aacf73fb6e4fab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal\" (UID: \"28d643209d9562fc33aacf73fb6e4fab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.105840 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.105802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea773fa8591c47f0caeb4b4d0061f7eb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-138.ec2.internal\" (UID: \"ea773fa8591c47f0caeb4b4d0061f7eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.105840 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.105825 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28d643209d9562fc33aacf73fb6e4fab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal\" (UID: \"28d643209d9562fc33aacf73fb6e4fab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.105840 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.105835 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea773fa8591c47f0caeb4b4d0061f7eb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-138.ec2.internal\" (UID: \"ea773fa8591c47f0caeb4b4d0061f7eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.205472 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.205440 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.265250 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.265231 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.266435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.266420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.305891 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.305869 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.410123 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.410095 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.511058 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.511035 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.541499 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.541471 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 11:40:12.541917 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.541635 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:40:12.587617 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.587593 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:40:12.611759 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.611734 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.614109 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.614075 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:35:12 +0000 UTC" deadline="2027-12-26 03:08:48.927870207 +0000 UTC" Jul 07 11:40:12.614109 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.614107 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12879h28m36.313767226s" Jul 07 11:40:12.712366 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.712312 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.767561 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.767543 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:40:12.812428 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:12.812404 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-138.ec2.internal\" not found" Jul 07 11:40:12.848332 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:12.848300 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d643209d9562fc33aacf73fb6e4fab.slice/crio-829ef8060afacf51812019191a1f5d90be68a1ceb3ee9f17c0475d8967241dff WatchSource:0}: Error finding container 829ef8060afacf51812019191a1f5d90be68a1ceb3ee9f17c0475d8967241dff: Status 404 returned error can't find the container with id 829ef8060afacf51812019191a1f5d90be68a1ceb3ee9f17c0475d8967241dff Jul 07 11:40:12.848517 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:12.848498 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea773fa8591c47f0caeb4b4d0061f7eb.slice/crio-50d393b13e2a84b52c5e119733c2fca4d79db87c3be929a0e6ac5672be797b12 WatchSource:0}: Error finding container 50d393b13e2a84b52c5e119733c2fca4d79db87c3be929a0e6ac5672be797b12: Status 404 returned error can't find the container with id 50d393b13e2a84b52c5e119733c2fca4d79db87c3be929a0e6ac5672be797b12 Jul 07 11:40:12.852484 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.852469 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:40:12.874176 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.874153 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:40:12.903854 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.903832 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.918048 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.918028 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:40:12.919077 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.919060 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" Jul 07 11:40:12.928708 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:12.928692 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:40:13.353527 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.353499 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:40:13.582190 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.582160 2580 apiserver.go:52] "Watching apiserver" Jul 07 11:40:13.588078 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.588043 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 11:40:13.588673 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.588631 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-cgbd2","openshift-multus/multus-gqrt9","openshift-multus/network-metrics-daemon-2ml8h","openshift-network-diagnostics/network-check-target-qdr6z","kube-system/konnectivity-agent-mhtgj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7","openshift-cluster-node-tuning-operator/tuned-2d92w","openshift-dns/node-resolver-rpxgg","openshift-image-registry/node-ca-ztk6q","openshift-network-operator/iptables-alerter-7gww9","openshift-ovn-kubernetes/ovnkube-node-2c7qq","kube-system/global-pull-secret-syncer-v7v7r","kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal"] Jul 07 11:40:13.590207 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.590180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.591600 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.591576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.594173 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594109 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:13.594173 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594119 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:13.594340 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.594215 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:13.594340 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.594242 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:13.595215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594836 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 11:40:13.595215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 11:40:13.595215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594903 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 11:40:13.595215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594927 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 11:40:13.595215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594990 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tjp8c\"" Jul 07 11:40:13.595215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.594905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.595577 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.595371 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.595577 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.595423 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m4m9z\"" Jul 07 11:40:13.597520 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.597498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.598525 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.598220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.599385 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.599366 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 11:40:13.600047 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.599801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.600161 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.600145 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xmn67\"" Jul 07 11:40:13.600438 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.600274 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.600438 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.600299 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 11:40:13.600597 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.600515 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 11:40:13.601383 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.601358 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.603188 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.602901 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.603188 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.603111 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cm68k\"" Jul 07 11:40:13.603420 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.603361 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.603420 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.603377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lf7bl\"" Jul 07 11:40:13.603778 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.603726 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.603893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.603824 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.603959 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.603919 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jfcw7\"" Jul 07 11:40:13.604061 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.604038 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 11:40:13.604118 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.604107 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.604253 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.604228 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.605376 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.605357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.607234 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.607213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.607777 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.607757 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.608014 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.607975 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 11:40:13.608156 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608134 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b6bq8\"" Jul 07 11:40:13.608156 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608152 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.608301 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.608301 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608222 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zksct\"" Jul 07 11:40:13.608301 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.608283 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:13.608433 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608136 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.608433 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608050 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.608433 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.608135 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 11:40:13.609145 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.609129 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 11:40:13.610805 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.610785 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x7b9p\"" Jul 07 11:40:13.610805 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.610806 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 11:40:13.610967 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.610887 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 11:40:13.611137 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.610989 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 11:40:13.611513 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.611402 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 11:40:13.611513 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.611488 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 11:40:13.616917 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.616896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.617027 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.616925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.617027 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.616977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-device-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.617027 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.616998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-modprobe-d\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.617027 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-systemd\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-run\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617057 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-conf-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-registration-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0fb452a-c51e-4c8b-9527-c23456b6e4cb-konnectivity-ca\") pod \"konnectivity-agent-mhtgj\" (UID: \"c0fb452a-c51e-4c8b-9527-c23456b6e4cb\") " pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvpsq\" (UniqueName: \"kubernetes.io/projected/560b0eae-753a-4a59-83d4-906a93f8094f-kube-api-access-wvpsq\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lrdw\" (UniqueName: \"kubernetes.io/projected/28111496-14cc-4e90-ae0c-88d61f489e9b-kube-api-access-4lrdw\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-hostroot\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:13.617278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-host\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9t9g\" (UniqueName: \"kubernetes.io/projected/b9d58c15-95ca-4172-a2ab-c76893c6ebea-kube-api-access-x9t9g\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-system-cni-dir\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55207ce8-d0ca-4d60-a966-daa4e9ecca81-hosts-file\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28111496-14cc-4e90-ae0c-88d61f489e9b-iptables-alerter-script\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-cnibin\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bxtt\" (UniqueName: \"kubernetes.io/projected/529387f7-20c3-4ce0-aa48-3861845232aa-kube-api-access-9bxtt\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-socket-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9d58c15-95ca-4172-a2ab-c76893c6ebea-host\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617543 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cnibin\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt758\" (UniqueName: \"kubernetes.io/projected/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-kube-api-access-qt758\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617584 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-daemon-config\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-var-lib-kubelet\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9d58c15-95ca-4172-a2ab-c76893c6ebea-serviceca\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.617631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28111496-14cc-4e90-ae0c-88d61f489e9b-host-slash\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0fb452a-c51e-4c8b-9527-c23456b6e4cb-agent-certs\") pod \"konnectivity-agent-mhtgj\" (UID: \"c0fb452a-c51e-4c8b-9527-c23456b6e4cb\") " pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysctl-d\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-cni-bin\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc7fm\" (UniqueName: \"kubernetes.io/projected/b5f90c71-15d1-435d-82a2-dec60a9655f5-kube-api-access-pc7fm\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55207ce8-d0ca-4d60-a966-daa4e9ecca81-tmp-dir\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.617975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-kubernetes\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysctl-conf\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-tmp\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysconfig\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-lib-modules\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-socket-dir-parent\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbvgr\" (UniqueName: \"kubernetes.io/projected/55207ce8-d0ca-4d60-a966-daa4e9ecca81-kube-api-access-cbvgr\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618317 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-netns\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-multus-certs\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.618427 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-sys-fs\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-cni-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618443 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-sys\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sztl\" (UniqueName: \"kubernetes.io/projected/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-kube-api-access-6sztl\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-system-cni-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618523 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-k8s-cni-cncf-io\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-kubelet\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618568 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-etc-kubernetes\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-tuned\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-os-release\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-os-release\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5f90c71-15d1-435d-82a2-dec60a9655f5-cni-binary-copy\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.619441 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.618819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-cni-multus\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.623494 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.623315 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:40:13.646184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.646148 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zwwhk" Jul 07 11:40:13.655137 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.655117 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zwwhk" Jul 07 11:40:13.704603 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.704580 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 11:40:13.719646 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-sys-fs\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.719646 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-cni-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-cni-netd\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-sys\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sztl\" (UniqueName: \"kubernetes.io/projected/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-kube-api-access-6sztl\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-system-cni-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719787 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-sys-fs\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-k8s-cni-cncf-io\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-kubelet\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719829 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-sys\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.719849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-etc-kubernetes\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719861 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-system-cni-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-slash\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719881 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-cni-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-k8s-cni-cncf-io\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-tuned\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719964 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-kubelet\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-os-release\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.719949 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-etc-kubernetes\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-os-release\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5f90c71-15d1-435d-82a2-dec60a9655f5-cni-binary-copy\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-log-socket\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720072 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-os-release\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-ovnkube-config\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-env-overrides\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-cni-multus\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.720357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-os-release\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720272 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-cni-multus\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2375073f-e7ec-4841-9d69-cf8a98623320-ovn-node-metrics-cert\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-device-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-modprobe-d\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-systemd\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720592 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-run\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-conf-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720681 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-device-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720696 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-systemd\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720740 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-run\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720714 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5f90c71-15d1-435d-82a2-dec60a9655f5-cni-binary-copy\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.721241 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720781 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-conf-dir\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-kubelet\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-ovnkube-script-lib\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720849 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-modprobe-d\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-registration-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0fb452a-c51e-4c8b-9527-c23456b6e4cb-konnectivity-ca\") pod \"konnectivity-agent-mhtgj\" (UID: \"c0fb452a-c51e-4c8b-9527-c23456b6e4cb\") " pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-registration-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvpsq\" (UniqueName: \"kubernetes.io/projected/560b0eae-753a-4a59-83d4-906a93f8094f-kube-api-access-wvpsq\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.720992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lrdw\" (UniqueName: \"kubernetes.io/projected/28111496-14cc-4e90-ae0c-88d61f489e9b-kube-api-access-4lrdw\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721018 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-hostroot\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-run-ovn-kubernetes\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a31e3c1-5122-433a-88e1-da067af93778-dbus\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721121 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-hostroot\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:13.722171 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cni-binary-copy\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.721245 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-host\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.721345 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:40:14.221307937 +0000 UTC m=+3.065032036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721372 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-host\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9t9g\" (UniqueName: \"kubernetes.io/projected/b9d58c15-95ca-4172-a2ab-c76893c6ebea-kube-api-access-x9t9g\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-system-cni-dir\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-systemd\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-system-cni-dir\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-var-lib-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721540 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c0fb452a-c51e-4c8b-9527-c23456b6e4cb-konnectivity-ca\") pod \"konnectivity-agent-mhtgj\" (UID: \"c0fb452a-c51e-4c8b-9527-c23456b6e4cb\") " pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a31e3c1-5122-433a-88e1-da067af93778-kubelet-config\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55207ce8-d0ca-4d60-a966-daa4e9ecca81-hosts-file\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28111496-14cc-4e90-ae0c-88d61f489e9b-iptables-alerter-script\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-cnibin\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-run-netns\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.723127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bxtt\" (UniqueName: \"kubernetes.io/projected/529387f7-20c3-4ce0-aa48-3861845232aa-kube-api-access-9bxtt\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721787 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721845 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55207ce8-d0ca-4d60-a966-daa4e9ecca81-hosts-file\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-socket-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-socket-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.721970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9d58c15-95ca-4172-a2ab-c76893c6ebea-host\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cnibin\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-cnibin\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt758\" (UniqueName: \"kubernetes.io/projected/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-kube-api-access-qt758\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-daemon-config\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-cni-bin\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc286\" (UniqueName: \"kubernetes.io/projected/2375073f-e7ec-4841-9d69-cf8a98623320-kube-api-access-kc286\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-var-lib-kubelet\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9d58c15-95ca-4172-a2ab-c76893c6ebea-serviceca\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28111496-14cc-4e90-ae0c-88d61f489e9b-host-slash\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-etc-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0fb452a-c51e-4c8b-9527-c23456b6e4cb-agent-certs\") pod \"konnectivity-agent-mhtgj\" (UID: \"c0fb452a-c51e-4c8b-9527-c23456b6e4cb\") " pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.724088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722308 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28111496-14cc-4e90-ae0c-88d61f489e9b-host-slash\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysctl-d\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722399 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28111496-14cc-4e90-ae0c-88d61f489e9b-iptables-alerter-script\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-cni-bin\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc7fm\" (UniqueName: \"kubernetes.io/projected/b5f90c71-15d1-435d-82a2-dec60a9655f5-kube-api-access-pc7fm\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9d58c15-95ca-4172-a2ab-c76893c6ebea-host\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722533 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-var-lib-kubelet\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722557 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-ovn\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-node-log\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722629 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9d58c15-95ca-4172-a2ab-c76893c6ebea-serviceca\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55207ce8-d0ca-4d60-a966-daa4e9ecca81-tmp-dir\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-kubernetes\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-daemon-config\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.724924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/560b0eae-753a-4a59-83d4-906a93f8094f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-cnibin\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722793 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysctl-d\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-var-lib-cni-bin\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysctl-conf\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722839 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-kubernetes\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-tmp\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysconfig\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.722999 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysctl-conf\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-lib-modules\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723020 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55207ce8-d0ca-4d60-a966-daa4e9ecca81-tmp-dir\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723074 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-sysconfig\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723109 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-lib-modules\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-socket-dir-parent\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbvgr\" (UniqueName: \"kubernetes.io/projected/55207ce8-d0ca-4d60-a966-daa4e9ecca81-kube-api-access-cbvgr\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-netns\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723212 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-multus-socket-dir-parent\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-multus-certs\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.725752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-systemd-units\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.726486 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-multus-certs\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.726486 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.723333 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5f90c71-15d1-435d-82a2-dec60a9655f5-host-run-netns\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.726486 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.725828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-tmp\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.726486 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.725979 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-etc-tuned\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.726486 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.726092 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c0fb452a-c51e-4c8b-9527-c23456b6e4cb-agent-certs\") pod \"konnectivity-agent-mhtgj\" (UID: \"c0fb452a-c51e-4c8b-9527-c23456b6e4cb\") " pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.732975 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.732948 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:13.732975 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.732974 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:13.733145 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.732987 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:13.733145 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.733047 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:14.233028191 +0000 UTC m=+3.076752295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:13.736480 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.736426 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvpsq\" (UniqueName: \"kubernetes.io/projected/560b0eae-753a-4a59-83d4-906a93f8094f-kube-api-access-wvpsq\") pod \"aws-ebs-csi-driver-node-nnkb7\" (UID: \"560b0eae-753a-4a59-83d4-906a93f8094f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.737770 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.737744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bxtt\" (UniqueName: \"kubernetes.io/projected/529387f7-20c3-4ce0-aa48-3861845232aa-kube-api-access-9bxtt\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:13.737875 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.737793 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lrdw\" (UniqueName: \"kubernetes.io/projected/28111496-14cc-4e90-ae0c-88d61f489e9b-kube-api-access-4lrdw\") pod \"iptables-alerter-7gww9\" (UID: \"28111496-14cc-4e90-ae0c-88d61f489e9b\") " pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.737875 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.737852 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9t9g\" (UniqueName: \"kubernetes.io/projected/b9d58c15-95ca-4172-a2ab-c76893c6ebea-kube-api-access-x9t9g\") pod \"node-ca-ztk6q\" (UID: \"b9d58c15-95ca-4172-a2ab-c76893c6ebea\") " pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.738351 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.738302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sztl\" (UniqueName: \"kubernetes.io/projected/e2e5b17d-5bd6-4eb1-8b04-d331500eab5d-kube-api-access-6sztl\") pod \"tuned-2d92w\" (UID: \"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d\") " pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.738822 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.738789 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc7fm\" (UniqueName: \"kubernetes.io/projected/b5f90c71-15d1-435d-82a2-dec60a9655f5-kube-api-access-pc7fm\") pod \"multus-gqrt9\" (UID: \"b5f90c71-15d1-435d-82a2-dec60a9655f5\") " pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.739591 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.739563 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbvgr\" (UniqueName: \"kubernetes.io/projected/55207ce8-d0ca-4d60-a966-daa4e9ecca81-kube-api-access-cbvgr\") pod \"node-resolver-rpxgg\" (UID: \"55207ce8-d0ca-4d60-a966-daa4e9ecca81\") " pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.739853 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.739806 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt758\" (UniqueName: \"kubernetes.io/projected/b5c6c0cc-9fe5-4237-a5a5-2156991897d5-kube-api-access-qt758\") pod \"multus-additional-cni-plugins-cgbd2\" (UID: \"b5c6c0cc-9fe5-4237-a5a5-2156991897d5\") " pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.824287 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-systemd-units\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-cni-netd\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-slash\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.824455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-log-socket\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-systemd-units\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-ovnkube-config\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-slash\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-env-overrides\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2375073f-e7ec-4841-9d69-cf8a98623320-ovn-node-metrics-cert\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-kubelet\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-ovnkube-script-lib\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-run-ovn-kubernetes\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824673 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a31e3c1-5122-433a-88e1-da067af93778-dbus\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-cni-netd\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-systemd\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824765 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a31e3c1-5122-433a-88e1-da067af93778-dbus\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.824792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-var-lib-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a31e3c1-5122-433a-88e1-da067af93778-kubelet-config\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-run-netns\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-cni-bin\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc286\" (UniqueName: \"kubernetes.io/projected/2375073f-e7ec-4841-9d69-cf8a98623320-kube-api-access-kc286\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-etc-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-ovn\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.824972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-node-log\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-ovnkube-config\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825060 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a31e3c1-5122-433a-88e1-da067af93778-kubelet-config\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-systemd\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-run-netns\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825150 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-cni-bin\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.825188 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825185 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-log-socket\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.825419 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:13.825246 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:14.325228136 +0000 UTC m=+3.168952233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825342 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-var-lib-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825430 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-etc-openvswitch\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825478 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-run-ovn\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825523 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-node-log\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825553 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-kubelet\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2375073f-e7ec-4841-9d69-cf8a98623320-host-run-ovn-kubernetes\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825762 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-ovnkube-script-lib\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.826231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.825879 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2375073f-e7ec-4841-9d69-cf8a98623320-env-overrides\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.827716 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.827692 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2375073f-e7ec-4841-9d69-cf8a98623320-ovn-node-metrics-cert\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.835081 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.835031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" event={"ID":"28d643209d9562fc33aacf73fb6e4fab","Type":"ContainerStarted","Data":"829ef8060afacf51812019191a1f5d90be68a1ceb3ee9f17c0475d8967241dff"} Jul 07 11:40:13.837302 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.837272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" event={"ID":"ea773fa8591c47f0caeb4b4d0061f7eb","Type":"ContainerStarted","Data":"50d393b13e2a84b52c5e119733c2fca4d79db87c3be929a0e6ac5672be797b12"} Jul 07 11:40:13.844493 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.844468 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc286\" (UniqueName: \"kubernetes.io/projected/2375073f-e7ec-4841-9d69-cf8a98623320-kube-api-access-kc286\") pod \"ovnkube-node-2c7qq\" (UID: \"2375073f-e7ec-4841-9d69-cf8a98623320\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:13.904903 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.904879 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" Jul 07 11:40:13.916136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.916110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gqrt9" Jul 07 11:40:13.924931 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.924912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:13.931314 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.931294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" Jul 07 11:40:13.943900 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.943881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2d92w" Jul 07 11:40:13.951435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.951416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rpxgg" Jul 07 11:40:13.962133 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.962115 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ztk6q" Jul 07 11:40:13.969068 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.969047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7gww9" Jul 07 11:40:13.975752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:13.975733 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:14.228377 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.228295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:14.228524 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.228456 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:14.228524 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.228521 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:40:15.228505262 +0000 UTC m=+4.072229342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:14.329438 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.329409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:14.329595 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.329475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:14.329595 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.329539 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:14.329722 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.329612 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:15.329596001 +0000 UTC m=+4.173320093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:14.329722 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.329627 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:14.329722 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.329654 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:14.329722 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.329671 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:14.329900 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.329737 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:15.329719354 +0000 UTC m=+4.173443440 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:14.657532 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.657497 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:35:14 +0000 UTC" deadline="2028-01-15 23:23:38.699413666 +0000 UTC" Jul 07 11:40:14.657532 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.657525 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13379h43m24.041892225s" Jul 07 11:40:14.786486 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.786450 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c6c0cc_9fe5_4237_a5a5_2156991897d5.slice/crio-66ca65fc084e103586fa37ae7fbd646acbf688c4814069991ebd4ce2b390e0a3 WatchSource:0}: Error finding container 66ca65fc084e103586fa37ae7fbd646acbf688c4814069991ebd4ce2b390e0a3: Status 404 returned error can't find the container with id 66ca65fc084e103586fa37ae7fbd646acbf688c4814069991ebd4ce2b390e0a3 Jul 07 11:40:14.789835 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.789815 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2375073f_e7ec_4841_9d69_cf8a98623320.slice/crio-609118635b7fafa08026d81d30756ab330b24c281747a48c05283dff0fefe4af WatchSource:0}: Error finding container 609118635b7fafa08026d81d30756ab330b24c281747a48c05283dff0fefe4af: Status 404 returned error can't find the container with id 609118635b7fafa08026d81d30756ab330b24c281747a48c05283dff0fefe4af Jul 07 11:40:14.792060 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.791989 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d58c15_95ca_4172_a2ab_c76893c6ebea.slice/crio-32996e0a48bafe682c5ea0bbe5aeda31f5ecd4e8f6fbc970ca26dcecc40419a6 WatchSource:0}: Error finding container 32996e0a48bafe682c5ea0bbe5aeda31f5ecd4e8f6fbc970ca26dcecc40419a6: Status 404 returned error can't find the container with id 32996e0a48bafe682c5ea0bbe5aeda31f5ecd4e8f6fbc970ca26dcecc40419a6 Jul 07 11:40:14.793263 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.793179 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28111496_14cc_4e90_ae0c_88d61f489e9b.slice/crio-e2517cf1a8b33fbbb4506342373c802b26867b2fa64b276db963590278bf466c WatchSource:0}: Error finding container e2517cf1a8b33fbbb4506342373c802b26867b2fa64b276db963590278bf466c: Status 404 returned error can't find the container with id e2517cf1a8b33fbbb4506342373c802b26867b2fa64b276db963590278bf466c Jul 07 11:40:14.795615 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.795010 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fb452a_c51e_4c8b_9527_c23456b6e4cb.slice/crio-e13c07b95ef2bb1d88bba64f36c55fdfaa9379b96d8b970aa8b7eb9c50a78e7a WatchSource:0}: Error finding container e13c07b95ef2bb1d88bba64f36c55fdfaa9379b96d8b970aa8b7eb9c50a78e7a: Status 404 returned error can't find the container with id e13c07b95ef2bb1d88bba64f36c55fdfaa9379b96d8b970aa8b7eb9c50a78e7a Jul 07 11:40:14.795848 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.795822 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560b0eae_753a_4a59_83d4_906a93f8094f.slice/crio-fec731b667e5dc4f6f07794975ea5fbcb551a7cf8e39454b384795ab9daf779f WatchSource:0}: Error finding container fec731b667e5dc4f6f07794975ea5fbcb551a7cf8e39454b384795ab9daf779f: Status 404 returned error can't find the container with id fec731b667e5dc4f6f07794975ea5fbcb551a7cf8e39454b384795ab9daf779f Jul 07 11:40:14.797150 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.796738 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e5b17d_5bd6_4eb1_8b04_d331500eab5d.slice/crio-362342652ab75a008706df1735463cfc3d1eea7634e82a1b70ccc2dff9934246 WatchSource:0}: Error finding container 362342652ab75a008706df1735463cfc3d1eea7634e82a1b70ccc2dff9934246: Status 404 returned error can't find the container with id 362342652ab75a008706df1735463cfc3d1eea7634e82a1b70ccc2dff9934246 Jul 07 11:40:14.798527 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.798459 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55207ce8_d0ca_4d60_a966_daa4e9ecca81.slice/crio-72a1c4d4c96896c0c0c7a67c80d2575911679504479ed2cb6cda5bf315dd37f2 WatchSource:0}: Error finding container 72a1c4d4c96896c0c0c7a67c80d2575911679504479ed2cb6cda5bf315dd37f2: Status 404 returned error can't find the container with id 72a1c4d4c96896c0c0c7a67c80d2575911679504479ed2cb6cda5bf315dd37f2 Jul 07 11:40:14.799172 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:14.799094 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f90c71_15d1_435d_82a2_dec60a9655f5.slice/crio-3cdb052e7b021e4574ea98ba905ce4128447bc65a46d09b2f28f855af64d06df WatchSource:0}: Error finding container 3cdb052e7b021e4574ea98ba905ce4128447bc65a46d09b2f28f855af64d06df: Status 404 returned error can't find the container with id 3cdb052e7b021e4574ea98ba905ce4128447bc65a46d09b2f28f855af64d06df Jul 07 11:40:14.829817 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.829791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:14.829902 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.829828 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:14.829970 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.829941 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:14.830134 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:14.830100 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:14.839677 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.839654 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" event={"ID":"560b0eae-753a-4a59-83d4-906a93f8094f","Type":"ContainerStarted","Data":"fec731b667e5dc4f6f07794975ea5fbcb551a7cf8e39454b384795ab9daf779f"} Jul 07 11:40:14.840561 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.840540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mhtgj" event={"ID":"c0fb452a-c51e-4c8b-9527-c23456b6e4cb","Type":"ContainerStarted","Data":"e13c07b95ef2bb1d88bba64f36c55fdfaa9379b96d8b970aa8b7eb9c50a78e7a"} Jul 07 11:40:14.841572 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.841550 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ztk6q" event={"ID":"b9d58c15-95ca-4172-a2ab-c76893c6ebea","Type":"ContainerStarted","Data":"32996e0a48bafe682c5ea0bbe5aeda31f5ecd4e8f6fbc970ca26dcecc40419a6"} Jul 07 11:40:14.842661 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.842639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"609118635b7fafa08026d81d30756ab330b24c281747a48c05283dff0fefe4af"} Jul 07 11:40:14.843889 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.843596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerStarted","Data":"66ca65fc084e103586fa37ae7fbd646acbf688c4814069991ebd4ce2b390e0a3"} Jul 07 11:40:14.846389 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.846360 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gqrt9" event={"ID":"b5f90c71-15d1-435d-82a2-dec60a9655f5","Type":"ContainerStarted","Data":"3cdb052e7b021e4574ea98ba905ce4128447bc65a46d09b2f28f855af64d06df"} Jul 07 11:40:14.848578 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.848551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rpxgg" event={"ID":"55207ce8-d0ca-4d60-a966-daa4e9ecca81","Type":"ContainerStarted","Data":"72a1c4d4c96896c0c0c7a67c80d2575911679504479ed2cb6cda5bf315dd37f2"} Jul 07 11:40:14.850126 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.850102 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7gww9" event={"ID":"28111496-14cc-4e90-ae0c-88d61f489e9b","Type":"ContainerStarted","Data":"e2517cf1a8b33fbbb4506342373c802b26867b2fa64b276db963590278bf466c"} Jul 07 11:40:14.851265 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:14.851242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2d92w" event={"ID":"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d","Type":"ContainerStarted","Data":"362342652ab75a008706df1735463cfc3d1eea7634e82a1b70ccc2dff9934246"} Jul 07 11:40:15.238712 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.237914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:15.238712 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.238174 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:15.238712 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.238258 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:40:17.238236992 +0000 UTC m=+6.081961072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:15.338993 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.338954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:15.339168 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.339039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:15.339242 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.339208 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:15.339328 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.339228 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:15.339328 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.339262 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:15.339328 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.339323 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:17.339303452 +0000 UTC m=+6.183027534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:15.339507 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.339411 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:15.339507 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.339446 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:17.339434269 +0000 UTC m=+6.183158352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:15.658034 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.657968 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:35:14 +0000 UTC" deadline="2028-01-13 01:42:21.426824819 +0000 UTC" Jul 07 11:40:15.658034 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.658035 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13310h2m5.768795334s" Jul 07 11:40:15.831947 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.831600 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:15.831947 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:15.831744 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:15.874975 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.874024 2580 generic.go:356] "Generic (PLEG): container finished" podID="28d643209d9562fc33aacf73fb6e4fab" containerID="8bde39dcac0234f3b92a9e9cfc7b55fb9ffeb1bd0fb6357ceed1140a9488d0cb" exitCode=0 Jul 07 11:40:15.874975 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.874144 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" event={"ID":"28d643209d9562fc33aacf73fb6e4fab","Type":"ContainerDied","Data":"8bde39dcac0234f3b92a9e9cfc7b55fb9ffeb1bd0fb6357ceed1140a9488d0cb"} Jul 07 11:40:15.885181 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:15.885110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" event={"ID":"ea773fa8591c47f0caeb4b4d0061f7eb","Type":"ContainerStarted","Data":"91b14ec94baef8c01d6f5ab424f3814b1333eb4a7873820a69dafd6ec7446ef4"} Jul 07 11:40:16.829486 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:16.829450 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:16.829994 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:16.829674 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:16.830158 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:16.830126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:16.830310 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:16.830240 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:16.896828 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:16.896782 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" event={"ID":"28d643209d9562fc33aacf73fb6e4fab","Type":"ContainerStarted","Data":"d9877d719c527f2be80ea0efd301f929c50d10bd51eeeb07fd9ccf231d66d963"} Jul 07 11:40:16.920268 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:16.919524 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-138.ec2.internal" podStartSLOduration=3.919501987 podStartE2EDuration="3.919501987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:15.937062375 +0000 UTC m=+4.780786479" watchObservedRunningTime="2026-07-07 11:40:16.919501987 +0000 UTC m=+5.763226092" Jul 07 11:40:17.257613 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:17.256983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:17.257613 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.257161 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:17.257613 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.257223 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:40:21.257205375 +0000 UTC m=+10.100929458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:17.357578 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:17.357541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:17.357739 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:17.357625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:17.357817 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.357799 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:17.357875 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.357821 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:17.357875 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.357835 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:17.357975 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.357895 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:21.357875993 +0000 UTC m=+10.201600075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:17.358437 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.358348 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:17.358437 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.358411 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:21.358393674 +0000 UTC m=+10.202117761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:17.831293 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:17.831249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:17.837599 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:17.837556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:18.829707 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:18.828969 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:18.829707 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:18.829101 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:18.829707 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:18.829554 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:18.829707 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:18.829666 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:19.830192 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:19.829595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:19.830192 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:19.829757 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:20.829506 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:20.829449 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:20.829687 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:20.829449 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:20.829687 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:20.829607 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:20.829816 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:20.829726 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:21.292249 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:21.291664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:21.292249 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.291815 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:21.292249 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.291883 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:40:29.291861801 +0000 UTC m=+18.135585884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:21.392567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:21.392644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.392777 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.392840 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:29.392820924 +0000 UTC m=+18.236545011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.393041 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.393064 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.393078 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:21.393200 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.393125 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:29.393110515 +0000 UTC m=+18.236834605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:21.831370 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:21.831259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:21.832015 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:21.831975 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:22.829888 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:22.829735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:22.829888 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:22.829869 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:22.830456 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:22.829741 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:22.830456 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:22.829969 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:23.830415 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:23.830380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:23.831048 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:23.830543 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:24.828770 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:24.828740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:24.828921 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:24.828879 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:24.828982 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:24.828923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:24.829026 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:24.829000 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:25.828932 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:25.828909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:25.829285 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:25.829021 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:26.829501 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:26.829464 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:26.829957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:26.829505 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:26.829957 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:26.829593 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:26.829957 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:26.829738 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:27.829125 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:27.829092 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:27.829305 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:27.829242 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:28.829209 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:28.829165 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:28.829788 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:28.829164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:28.829788 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:28.829320 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:28.829788 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:28.829404 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:29.352275 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:29.352240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:29.352426 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.352410 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:29.352497 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.352484 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:40:45.352458952 +0000 UTC m=+34.196183030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:29.453023 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:29.452990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:29.453208 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:29.453056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:29.453208 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.453157 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:29.453208 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.453171 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:29.453334 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.453211 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:29.453334 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.453225 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:29.453334 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.453229 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:45.453210979 +0000 UTC m=+34.296935066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:29.453334 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.453275 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:45.453256807 +0000 UTC m=+34.296980900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:29.829220 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:29.829180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:29.829566 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:29.829331 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:30.829353 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:30.829311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:30.829816 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:30.829362 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:30.829816 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:30.829453 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:30.829816 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:30.829576 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:31.830309 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:31.830276 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:31.830753 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:31.830390 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:32.829509 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:32.829477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:32.829509 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:32.829512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:32.829692 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:32.829593 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:32.829773 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:32.829743 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:33.828797 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:33.828771 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:33.829319 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:33.828906 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:34.829858 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.829687 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:34.830550 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.829737 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:34.830550 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:34.829951 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:34.830550 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:34.830018 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:34.932430 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.932380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2d92w" event={"ID":"e2e5b17d-5bd6-4eb1-8b04-d331500eab5d","Type":"ContainerStarted","Data":"f62c4aa6a75219bf3bc0ab40fdd56e46bb02b1632c3a18fdc8bba22e41436b42"} Jul 07 11:40:34.934119 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.934088 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" event={"ID":"560b0eae-753a-4a59-83d4-906a93f8094f","Type":"ContainerStarted","Data":"d3d95378fc8e28d4726d29e21d6c1568867be4ea4ac4e1cd3b8ee85ec3e55ab6"} Jul 07 11:40:34.935711 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.935685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mhtgj" event={"ID":"c0fb452a-c51e-4c8b-9527-c23456b6e4cb","Type":"ContainerStarted","Data":"4f947f82a5d0f1c1dcc146fb0eddffff3404daa3d15c90eb6ac16d28fc80c1dd"} Jul 07 11:40:34.937630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.937580 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ztk6q" event={"ID":"b9d58c15-95ca-4172-a2ab-c76893c6ebea","Type":"ContainerStarted","Data":"af475e7b0a06550834fb12afb4b0be3a3a819b4a5a95365f45161aa03fe8d618"} Jul 07 11:40:34.940821 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.940797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"667760da262cb36b1aea27a24becbcdc6459e9bdbcf0a33d7a7bc0f877e67e72"} Jul 07 11:40:34.940932 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.940865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"424a362d31df2aac8844facfc2603f7b6f7b549cb1d137686911698d5cc6f780"} Jul 07 11:40:34.940932 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.940902 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"95c9d497b676295e001dbcf7543da09c8c5aaf17dc2d7b407e06b48490adf757"} Jul 07 11:40:34.941042 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.940950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"1ee1ab0220621549589ba9b0fbc1bc2623bec56c8e2010ad4a3fba7135c56425"} Jul 07 11:40:34.941042 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.940986 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"4b6e3dcd3af344b645a4fd9c9a0720eb32522d637bec3ec19c73e8fcd26387c7"} Jul 07 11:40:34.941042 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.941021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"1acba9856c60dfc83a91834064df9d47b873cc21559147dc94a214ed84129185"} Jul 07 11:40:34.942456 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.942431 2580 generic.go:356] "Generic (PLEG): container finished" podID="b5c6c0cc-9fe5-4237-a5a5-2156991897d5" containerID="798baed8eb0a0fe262e479a96448d102d23a0e65b005d75413d56c7e5fef79ae" exitCode=0 Jul 07 11:40:34.942545 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.942513 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerDied","Data":"798baed8eb0a0fe262e479a96448d102d23a0e65b005d75413d56c7e5fef79ae"} Jul 07 11:40:34.944120 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.944099 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gqrt9" event={"ID":"b5f90c71-15d1-435d-82a2-dec60a9655f5","Type":"ContainerStarted","Data":"322511c1441a25fea24dd60c251795fedd9e2b4b932401b7cfa2270b49fc1f77"} Jul 07 11:40:34.945859 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.945837 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rpxgg" event={"ID":"55207ce8-d0ca-4d60-a966-daa4e9ecca81","Type":"ContainerStarted","Data":"5927d66cbc97cdfb14de12c74b4deaa6772c993ec3010619cd1867b31af64cbb"} Jul 07 11:40:34.949894 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.949857 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-138.ec2.internal" podStartSLOduration=21.949841845999998 podStartE2EDuration="21.949841846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:16.92020835 +0000 UTC m=+5.763932457" watchObservedRunningTime="2026-07-07 11:40:34.949841846 +0000 UTC m=+23.793565949" Jul 07 11:40:34.962893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.962856 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2d92w" podStartSLOduration=3.78252656 podStartE2EDuration="22.962841441s" totalImagesPullingTime="19.180314881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:34.949427471 +0000 UTC m=+23.793151573" watchObservedRunningTime="2026-07-07 11:40:34.962841441 +0000 UTC m=+23.806565542" Jul 07 11:40:34.984220 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.984153 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mhtgj" podStartSLOduration=7.970212428 podStartE2EDuration="22.984143099s" totalImagesPullingTime="15.013930671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:34.967410178 +0000 UTC m=+23.811134277" watchObservedRunningTime="2026-07-07 11:40:34.984143099 +0000 UTC m=+23.827867202" Jul 07 11:40:34.984423 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.984399 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rpxgg" podStartSLOduration=3.828683176 podStartE2EDuration="22.984389404s" totalImagesPullingTime="19.155706228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:34.984315403 +0000 UTC m=+23.828039484" watchObservedRunningTime="2026-07-07 11:40:34.984389404 +0000 UTC m=+23.828113504" Jul 07 11:40:34.998329 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:34.998299 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ztk6q" podStartSLOduration=3.834672161 podStartE2EDuration="22.998288812s" totalImagesPullingTime="19.163616651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:34.998145539 +0000 UTC m=+23.841869641" watchObservedRunningTime="2026-07-07 11:40:34.998288812 +0000 UTC m=+23.842012912" Jul 07 11:40:35.042089 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.042058 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gqrt9" podStartSLOduration=3.833857567 podStartE2EDuration="23.04204598s" totalImagesPullingTime="19.208188413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:35.041915093 +0000 UTC m=+23.885639193" watchObservedRunningTime="2026-07-07 11:40:35.04204598 +0000 UTC m=+23.885770057" Jul 07 11:40:35.316313 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.316282 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 11:40:35.682455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.682367 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:40:35.316310607Z","UUID":"79238327-6a6d-41e0-9d51-495ac85c12c7","Handler":null,"Name":"","Endpoint":""} Jul 07 11:40:35.683876 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.683855 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 11:40:35.683995 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.683889 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 11:40:35.752510 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.752482 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:35.753089 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.753070 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:35.829470 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.829437 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:35.829609 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:35.829581 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:35.951162 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.951067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" event={"ID":"560b0eae-753a-4a59-83d4-906a93f8094f","Type":"ContainerStarted","Data":"5ba5c63bbeb5d17c6c42606f7a7921fdd8d6f269156b0a0d537e76869b571234"} Jul 07 11:40:35.953949 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.953591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7gww9" event={"ID":"28111496-14cc-4e90-ae0c-88d61f489e9b","Type":"ContainerStarted","Data":"fad69dadc9ab3b44953e6b7c7cbc3bc32bde3d36763d184281282b1a80607bb6"} Jul 07 11:40:35.953949 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.953674 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:35.954247 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.954224 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mhtgj" Jul 07 11:40:35.971187 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:35.971128 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7gww9" podStartSLOduration=4.810674912 podStartE2EDuration="23.97111046s" totalImagesPullingTime="19.160435548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:35.97046345 +0000 UTC m=+24.814187552" watchObservedRunningTime="2026-07-07 11:40:35.97111046 +0000 UTC m=+24.814834561" Jul 07 11:40:36.829019 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:36.828961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:36.829019 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:36.828998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:36.829178 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:36.829084 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:36.829309 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:36.829274 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:36.957519 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:36.957483 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" event={"ID":"560b0eae-753a-4a59-83d4-906a93f8094f","Type":"ContainerStarted","Data":"f086dd4cd18deabdec4518e76d559c432ecaf873c82a0b119edf887570cae84f"} Jul 07 11:40:36.962388 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:36.962355 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"38d1d4b8e3a6ff14f39a51f7d2d059f705602f27635539514b581f6753600e1d"} Jul 07 11:40:36.979679 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:36.979631 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnkb7" podStartSLOduration=3.526130873 podStartE2EDuration="24.979611361s" totalImagesPullingTime="21.453480488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:36.979146842 +0000 UTC m=+25.822870939" watchObservedRunningTime="2026-07-07 11:40:36.979611361 +0000 UTC m=+25.823335463" Jul 07 11:40:37.829522 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:37.829491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:37.829701 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:37.829636 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:38.829124 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:38.828932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:38.829578 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:38.828932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:38.829578 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:38.829214 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:38.829578 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:38.829296 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:39.829778 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:39.829708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:39.830480 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:39.829819 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:39.973521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:39.973483 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" event={"ID":"2375073f-e7ec-4841-9d69-cf8a98623320","Type":"ContainerStarted","Data":"47cb2e9936141d61b824ecb4017927204adbfc939294f8e3c356bae6fad17153"} Jul 07 11:40:39.973805 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:39.973700 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:39.975455 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:39.975293 2580 generic.go:356] "Generic (PLEG): container finished" podID="b5c6c0cc-9fe5-4237-a5a5-2156991897d5" containerID="56f6927456c2c20e70a1fb0d7a4414c5b016f3cf5dac9d44d8665bdb9db7cee5" exitCode=0 Jul 07 11:40:39.975583 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:39.975323 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerDied","Data":"56f6927456c2c20e70a1fb0d7a4414c5b016f3cf5dac9d44d8665bdb9db7cee5"} Jul 07 11:40:39.989079 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:39.989061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:40.006578 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:40.006543 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" podStartSLOduration=8.760406816 podStartE2EDuration="28.006530683s" totalImagesPullingTime="19.246123867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:40.004504171 +0000 UTC m=+28.848228272" watchObservedRunningTime="2026-07-07 11:40:40.006530683 +0000 UTC m=+28.850254783" Jul 07 11:40:40.829537 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:40.829498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:40.829777 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:40.829498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:40.829777 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:40.829623 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:40.829777 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:40.829648 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:40.977645 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:40.977621 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:40.977802 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:40.977688 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:40.991903 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:40.991881 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:40:41.555304 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.555278 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2ml8h"] Jul 07 11:40:41.555489 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.555394 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:41.555532 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:41.555487 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:41.555929 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.555911 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qdr6z"] Jul 07 11:40:41.555996 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.555986 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:41.556078 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:41.556061 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:41.586544 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.586515 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v7v7r"] Jul 07 11:40:41.586670 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.586600 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:41.586732 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:41.586688 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:41.981110 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.981078 2580 generic.go:356] "Generic (PLEG): container finished" podID="b5c6c0cc-9fe5-4237-a5a5-2156991897d5" containerID="a32871ebced9eba01c5870b3471d33ff702d3529eea6367b10268a6d06f46651" exitCode=0 Jul 07 11:40:41.981620 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:41.981161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerDied","Data":"a32871ebced9eba01c5870b3471d33ff702d3529eea6367b10268a6d06f46651"} Jul 07 11:40:42.829087 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:42.829058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:42.829256 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:42.829158 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:43.829356 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:43.829280 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:43.829356 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:43.829295 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:43.829767 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:43.829453 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:43.829767 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:43.829525 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:43.988833 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:43.988807 2580 generic.go:356] "Generic (PLEG): container finished" podID="b5c6c0cc-9fe5-4237-a5a5-2156991897d5" containerID="35efdf8a594bc21275e8feb1241183bba2317c29a372ce0ed2293c3f51dac481" exitCode=0 Jul 07 11:40:43.988996 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:43.988839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerDied","Data":"35efdf8a594bc21275e8feb1241183bba2317c29a372ce0ed2293c3f51dac481"} Jul 07 11:40:44.829980 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:44.829782 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:44.830442 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:44.830073 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:45.376354 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:45.376323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:45.376512 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.376448 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:45.376512 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.376500 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs podName:529387f7-20c3-4ce0-aa48-3861845232aa nodeName:}" failed. No retries permitted until 2026-07-07 11:41:17.3764866 +0000 UTC m=+66.220210678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs") pod "network-metrics-daemon-2ml8h" (UID: "529387f7-20c3-4ce0-aa48-3861845232aa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:40:45.476649 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:45.476619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:45.476824 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:45.476674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:45.476824 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.476781 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:45.476824 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.476812 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:40:45.476926 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.476837 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret podName:1a31e3c1-5122-433a-88e1-da067af93778 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:17.476822925 +0000 UTC m=+66.320547004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret") pod "global-pull-secret-syncer-v7v7r" (UID: "1a31e3c1-5122-433a-88e1-da067af93778") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:40:45.476926 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.476841 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:40:45.476926 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.476855 2580 projected.go:196] Error preparing data for projected volume kube-api-access-9m6sx for pod openshift-network-diagnostics/network-check-target-qdr6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:45.476926 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.476905 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx podName:f0a98300-a855-4d77-a89c-100c5e06e501 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:17.476888243 +0000 UTC m=+66.320612328 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9m6sx" (UniqueName: "kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx") pod "network-check-target-qdr6z" (UID: "f0a98300-a855-4d77-a89c-100c5e06e501") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:40:45.829779 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:45.829702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:45.829937 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.829817 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qdr6z" podUID="f0a98300-a855-4d77-a89c-100c5e06e501" Jul 07 11:40:45.829937 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:45.829875 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:45.830039 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:45.829985 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v7v7r" podUID="1a31e3c1-5122-433a-88e1-da067af93778" Jul 07 11:40:46.828837 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.828763 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:46.828996 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:46.828872 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2ml8h" podUID="529387f7-20c3-4ce0-aa48-3861845232aa" Jul 07 11:40:46.930547 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.930517 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-138.ec2.internal" event="NodeReady" Jul 07 11:40:46.931004 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.930669 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 11:40:46.989183 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.989154 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7766875f79-zmv45"] Jul 07 11:40:46.991603 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.991585 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:46.995375 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.995351 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xbhxf\"" Jul 07 11:40:46.995492 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.995352 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 11:40:46.995600 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.995578 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 11:40:46.995917 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:46.995898 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 11:40:47.004524 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.004502 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 11:40:47.021834 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.021812 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wcrp5"] Jul 07 11:40:47.023670 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.023652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.023860 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.023837 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7766875f79-zmv45"] Jul 07 11:40:47.025548 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.025531 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 11:40:47.026075 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.026060 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 11:40:47.026169 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.026128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 11:40:47.026398 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.026383 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-svb8z\"" Jul 07 11:40:47.032366 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.032188 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kc454"] Jul 07 11:40:47.036322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.034085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.038233 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.038190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 11:40:47.038327 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.038259 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 11:40:47.038327 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.038293 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4khfw\"" Jul 07 11:40:47.044473 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.044455 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wcrp5"] Jul 07 11:40:47.056363 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.056346 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kc454"] Jul 07 11:40:47.089509 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089637 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-installation-pull-secrets\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089637 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-certificates\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089637 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f7a8dfd-113d-4e30-9a4a-927004d0e035-ca-trust-extracted\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089767 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-trusted-ca\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089767 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-image-registry-private-configuration\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089767 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf85b\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-kube-api-access-mf85b\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.089767 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.089756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-bound-sa-token\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.190859 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.190825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-installation-pull-secrets\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.190859 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.190871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64pqs\" (UniqueName: \"kubernetes.io/projected/4aac21db-70cd-4e10-b7ff-700f56b241ba-kube-api-access-64pqs\") pod \"ingress-canary-wcrp5\" (UID: \"4aac21db-70cd-4e10-b7ff-700f56b241ba\") " pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.191115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.190892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tplgh\" (UniqueName: \"kubernetes.io/projected/8ebddf9a-a072-4c3c-a78e-047ae767d096-kube-api-access-tplgh\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.191115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.190919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-certificates\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.190975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f7a8dfd-113d-4e30-9a4a-927004d0e035-ca-trust-extracted\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-trusted-ca\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4aac21db-70cd-4e10-b7ff-700f56b241ba-cert\") pod \"ingress-canary-wcrp5\" (UID: \"4aac21db-70cd-4e10-b7ff-700f56b241ba\") " pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.191115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ebddf9a-a072-4c3c-a78e-047ae767d096-tmp-dir\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.191435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-image-registry-private-configuration\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf85b\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-kube-api-access-mf85b\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.191435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-bound-sa-token\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191435 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ebddf9a-a072-4c3c-a78e-047ae767d096-config-volume\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.191727 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191483 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f7a8dfd-113d-4e30-9a4a-927004d0e035-ca-trust-extracted\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191727 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.191570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-certificates\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.191727 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.191697 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 11:40:47.191870 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.191730 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7766875f79-zmv45: secret "image-registry-tls" not found Jul 07 11:40:47.191870 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.191813 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls podName:0f7a8dfd-113d-4e30-9a4a-927004d0e035 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:47.691794687 +0000 UTC m=+36.535518784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls") pod "image-registry-7766875f79-zmv45" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035") : secret "image-registry-tls" not found Jul 07 11:40:47.192051 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.192028 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-trusted-ca\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.195442 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.195414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-installation-pull-secrets\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.195529 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.195461 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-image-registry-private-configuration\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.206159 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.206135 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf85b\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-kube-api-access-mf85b\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.206844 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.206825 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-bound-sa-token\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.291994 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.291964 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.292145 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292013 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ebddf9a-a072-4c3c-a78e-047ae767d096-config-volume\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.292207 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.292145 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:40:47.292262 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.292252 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls podName:8ebddf9a-a072-4c3c-a78e-047ae767d096 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:47.792229182 +0000 UTC m=+36.635953279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls") pod "dns-default-kc454" (UID: "8ebddf9a-a072-4c3c-a78e-047ae767d096") : secret "dns-default-metrics-tls" not found Jul 07 11:40:47.292358 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64pqs\" (UniqueName: \"kubernetes.io/projected/4aac21db-70cd-4e10-b7ff-700f56b241ba-kube-api-access-64pqs\") pod \"ingress-canary-wcrp5\" (UID: \"4aac21db-70cd-4e10-b7ff-700f56b241ba\") " pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.292443 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tplgh\" (UniqueName: \"kubernetes.io/projected/8ebddf9a-a072-4c3c-a78e-047ae767d096-kube-api-access-tplgh\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.292443 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4aac21db-70cd-4e10-b7ff-700f56b241ba-cert\") pod \"ingress-canary-wcrp5\" (UID: \"4aac21db-70cd-4e10-b7ff-700f56b241ba\") " pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.292443 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ebddf9a-a072-4c3c-a78e-047ae767d096-tmp-dir\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.292627 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ebddf9a-a072-4c3c-a78e-047ae767d096-config-volume\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.292762 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.292746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ebddf9a-a072-4c3c-a78e-047ae767d096-tmp-dir\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.294932 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.294910 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4aac21db-70cd-4e10-b7ff-700f56b241ba-cert\") pod \"ingress-canary-wcrp5\" (UID: \"4aac21db-70cd-4e10-b7ff-700f56b241ba\") " pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.310127 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.310101 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tplgh\" (UniqueName: \"kubernetes.io/projected/8ebddf9a-a072-4c3c-a78e-047ae767d096-kube-api-access-tplgh\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.310297 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.310277 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64pqs\" (UniqueName: \"kubernetes.io/projected/4aac21db-70cd-4e10-b7ff-700f56b241ba-kube-api-access-64pqs\") pod \"ingress-canary-wcrp5\" (UID: \"4aac21db-70cd-4e10-b7ff-700f56b241ba\") " pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.334928 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.334902 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wcrp5" Jul 07 11:40:47.696086 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.696052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:47.696304 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.696243 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 11:40:47.696304 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.696265 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7766875f79-zmv45: secret "image-registry-tls" not found Jul 07 11:40:47.696397 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.696320 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls podName:0f7a8dfd-113d-4e30-9a4a-927004d0e035 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:48.696302165 +0000 UTC m=+37.540026248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls") pod "image-registry-7766875f79-zmv45" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035") : secret "image-registry-tls" not found Jul 07 11:40:47.797095 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.797062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:47.797305 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.797244 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:40:47.797374 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:47.797325 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls podName:8ebddf9a-a072-4c3c-a78e-047ae767d096 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:48.797304855 +0000 UTC m=+37.641028945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls") pod "dns-default-kc454" (UID: "8ebddf9a-a072-4c3c-a78e-047ae767d096") : secret "dns-default-metrics-tls" not found Jul 07 11:40:47.832030 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.832002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:40:47.832421 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.832403 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:40:47.834982 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.834741 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:40:47.834982 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.834958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vqglz\"" Jul 07 11:40:47.835311 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.835297 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:40:47.835479 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.835380 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:40:47.915302 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:47.915157 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wcrp5"] Jul 07 11:40:47.918303 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:47.918277 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aac21db_70cd_4e10_b7ff_700f56b241ba.slice/crio-72ec977f9f3153b10ec306fe2fb0ef11bb60c1cf1d82542e76af4ee62edb5098 WatchSource:0}: Error finding container 72ec977f9f3153b10ec306fe2fb0ef11bb60c1cf1d82542e76af4ee62edb5098: Status 404 returned error can't find the container with id 72ec977f9f3153b10ec306fe2fb0ef11bb60c1cf1d82542e76af4ee62edb5098 Jul 07 11:40:48.000521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.000484 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerStarted","Data":"c58c6b3bc2a175b29174afee0d837f408cfa3c46c18b29d12b7c5e0d6d04b6d2"} Jul 07 11:40:48.001557 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.001537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wcrp5" event={"ID":"4aac21db-70cd-4e10-b7ff-700f56b241ba","Type":"ContainerStarted","Data":"72ec977f9f3153b10ec306fe2fb0ef11bb60c1cf1d82542e76af4ee62edb5098"} Jul 07 11:40:48.703630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.703595 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:48.703788 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:48.703770 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 11:40:48.703826 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:48.703795 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7766875f79-zmv45: secret "image-registry-tls" not found Jul 07 11:40:48.703873 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:48.703859 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls podName:0f7a8dfd-113d-4e30-9a4a-927004d0e035 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:50.703839967 +0000 UTC m=+39.547564052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls") pod "image-registry-7766875f79-zmv45" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035") : secret "image-registry-tls" not found Jul 07 11:40:48.780630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.780598 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl"] Jul 07 11:40:48.782663 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.782637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" Jul 07 11:40:48.784734 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.784714 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-54gq5\"" Jul 07 11:40:48.785104 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.785087 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 11:40:48.785616 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.785595 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 11:40:48.799449 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.799428 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl"] Jul 07 11:40:48.804720 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.804700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:48.804876 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:48.804860 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:40:48.804952 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:48.804927 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls podName:8ebddf9a-a072-4c3c-a78e-047ae767d096 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:50.804906627 +0000 UTC m=+39.648630727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls") pod "dns-default-kc454" (UID: "8ebddf9a-a072-4c3c-a78e-047ae767d096") : secret "dns-default-metrics-tls" not found Jul 07 11:40:48.828885 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.828861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:40:48.830574 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.830554 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cprtm\"" Jul 07 11:40:48.833704 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.833690 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:40:48.877350 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.877328 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vlxns"] Jul 07 11:40:48.881054 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.881027 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:48.885151 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.885129 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 11:40:48.885262 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.885158 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 11:40:48.885719 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.885698 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 11:40:48.885825 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.885732 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f587k\"" Jul 07 11:40:48.885825 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.885746 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 11:40:48.905627 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.905590 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flzcc\" (UniqueName: \"kubernetes.io/projected/17b3e0f5-3744-471c-83a6-13588da0fdb8-kube-api-access-flzcc\") pod \"migrator-d57d58cc9-tlbfl\" (UID: \"17b3e0f5-3744-471c-83a6-13588da0fdb8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" Jul 07 11:40:48.906673 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:48.906645 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vlxns"] Jul 07 11:40:49.006031 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.005955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.006503 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.006096 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4855\" (UniqueName: \"kubernetes.io/projected/6583d603-c819-4182-a976-595b8fa41475-kube-api-access-j4855\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.006503 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.006146 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flzcc\" (UniqueName: \"kubernetes.io/projected/17b3e0f5-3744-471c-83a6-13588da0fdb8-kube-api-access-flzcc\") pod \"migrator-d57d58cc9-tlbfl\" (UID: \"17b3e0f5-3744-471c-83a6-13588da0fdb8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" Jul 07 11:40:49.006503 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.006215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6583d603-c819-4182-a976-595b8fa41475-data-volume\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.006503 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.006284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6583d603-c819-4182-a976-595b8fa41475-crio-socket\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.006503 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.006323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6583d603-c819-4182-a976-595b8fa41475-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.007003 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.006977 2580 generic.go:356] "Generic (PLEG): container finished" podID="b5c6c0cc-9fe5-4237-a5a5-2156991897d5" containerID="c58c6b3bc2a175b29174afee0d837f408cfa3c46c18b29d12b7c5e0d6d04b6d2" exitCode=0 Jul 07 11:40:49.007125 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.007024 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerDied","Data":"c58c6b3bc2a175b29174afee0d837f408cfa3c46c18b29d12b7c5e0d6d04b6d2"} Jul 07 11:40:49.020695 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.020673 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flzcc\" (UniqueName: \"kubernetes.io/projected/17b3e0f5-3744-471c-83a6-13588da0fdb8-kube-api-access-flzcc\") pod \"migrator-d57d58cc9-tlbfl\" (UID: \"17b3e0f5-3744-471c-83a6-13588da0fdb8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" Jul 07 11:40:49.094134 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.094106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" Jul 07 11:40:49.106827 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.106629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6583d603-c819-4182-a976-595b8fa41475-data-volume\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.106827 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.106703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6583d603-c819-4182-a976-595b8fa41475-crio-socket\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.106827 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.106776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6583d603-c819-4182-a976-595b8fa41475-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.106827 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.106804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.107607 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:49.107132 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 11:40:49.107607 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:49.107514 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6583d603-c819-4182-a976-595b8fa41475-kube-rbac-proxy-cm podName:6583d603-c819-4182-a976-595b8fa41475 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:49.607494049 +0000 UTC m=+38.451218127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6583d603-c819-4182-a976-595b8fa41475-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlxns" (UID: "6583d603-c819-4182-a976-595b8fa41475") : configmap "kube-rbac-proxy" not found Jul 07 11:40:49.107777 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:49.107361 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:40:49.107777 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:49.107720 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls podName:6583d603-c819-4182-a976-595b8fa41475 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:49.607703865 +0000 UTC m=+38.451427960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlxns" (UID: "6583d603-c819-4182-a976-595b8fa41475") : secret "insights-runtime-extractor-tls" not found Jul 07 11:40:49.107777 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.107360 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6583d603-c819-4182-a976-595b8fa41475-crio-socket\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.107939 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.107781 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6583d603-c819-4182-a976-595b8fa41475-data-volume\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.108055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.108030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4855\" (UniqueName: \"kubernetes.io/projected/6583d603-c819-4182-a976-595b8fa41475-kube-api-access-j4855\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.127287 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.127260 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4855\" (UniqueName: \"kubernetes.io/projected/6583d603-c819-4182-a976-595b8fa41475-kube-api-access-j4855\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.249675 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.249639 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl"] Jul 07 11:40:49.253761 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:40:49.253729 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b3e0f5_3744_471c_83a6_13588da0fdb8.slice/crio-0d545f94b824df00d0344046d2b2f621bc2fa9aa0adf87b1917fbb78e357b97c WatchSource:0}: Error finding container 0d545f94b824df00d0344046d2b2f621bc2fa9aa0adf87b1917fbb78e357b97c: Status 404 returned error can't find the container with id 0d545f94b824df00d0344046d2b2f621bc2fa9aa0adf87b1917fbb78e357b97c Jul 07 11:40:49.612682 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.612600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6583d603-c819-4182-a976-595b8fa41475-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.612841 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.612721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:49.612932 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:49.612902 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:40:49.613061 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:49.612990 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls podName:6583d603-c819-4182-a976-595b8fa41475 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:50.61296867 +0000 UTC m=+39.456692751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlxns" (UID: "6583d603-c819-4182-a976-595b8fa41475") : secret "insights-runtime-extractor-tls" not found Jul 07 11:40:49.613161 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:49.613136 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6583d603-c819-4182-a976-595b8fa41475-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:50.014230 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:50.014168 2580 generic.go:356] "Generic (PLEG): container finished" podID="b5c6c0cc-9fe5-4237-a5a5-2156991897d5" containerID="102556a3c5e1e8bee6f71a5d458ccaa9eb952ad477acad1056ab6c696fc73ffd" exitCode=0 Jul 07 11:40:50.014709 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:50.014302 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerDied","Data":"102556a3c5e1e8bee6f71a5d458ccaa9eb952ad477acad1056ab6c696fc73ffd"} Jul 07 11:40:50.015900 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:50.015860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" event={"ID":"17b3e0f5-3744-471c-83a6-13588da0fdb8","Type":"ContainerStarted","Data":"0d545f94b824df00d0344046d2b2f621bc2fa9aa0adf87b1917fbb78e357b97c"} Jul 07 11:40:50.620385 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:50.620140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:50.620553 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.620304 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:40:50.620553 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.620479 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls podName:6583d603-c819-4182-a976-595b8fa41475 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:52.620456327 +0000 UTC m=+41.464180422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlxns" (UID: "6583d603-c819-4182-a976-595b8fa41475") : secret "insights-runtime-extractor-tls" not found Jul 07 11:40:50.721581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:50.721548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:50.721732 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.721675 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 11:40:50.721732 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.721688 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7766875f79-zmv45: secret "image-registry-tls" not found Jul 07 11:40:50.721817 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.721734 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls podName:0f7a8dfd-113d-4e30-9a4a-927004d0e035 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:54.721721804 +0000 UTC m=+43.565445881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls") pod "image-registry-7766875f79-zmv45" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035") : secret "image-registry-tls" not found Jul 07 11:40:50.822922 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:50.822886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:50.823097 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.823034 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:40:50.823164 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:50.823107 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls podName:8ebddf9a-a072-4c3c-a78e-047ae767d096 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:54.823086882 +0000 UTC m=+43.666810962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls") pod "dns-default-kc454" (UID: "8ebddf9a-a072-4c3c-a78e-047ae767d096") : secret "dns-default-metrics-tls" not found Jul 07 11:40:51.021790 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:51.021761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" event={"ID":"b5c6c0cc-9fe5-4237-a5a5-2156991897d5","Type":"ContainerStarted","Data":"59be9d2c904ffba770ed62092be9ddcae6814199a374cbbea1b0236a590c3244"} Jul 07 11:40:51.023158 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:51.023133 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wcrp5" event={"ID":"4aac21db-70cd-4e10-b7ff-700f56b241ba","Type":"ContainerStarted","Data":"ac1ff332cc44714884c51f252e45cf2b729d6210284d5a71c462d6ebf6b203d3"} Jul 07 11:40:51.050720 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:51.050650 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cgbd2" podStartSLOduration=9.712994893 podStartE2EDuration="39.05063741s" totalImagesPullingTime="29.337642517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:51.050553382 +0000 UTC m=+39.894277482" watchObservedRunningTime="2026-07-07 11:40:51.05063741 +0000 UTC m=+39.894361509" Jul 07 11:40:51.071957 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:51.071897 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wcrp5" podStartSLOduration=2.595146433 podStartE2EDuration="5.071878244s" totalImagesPullingTime="2.476731811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:51.071129998 +0000 UTC m=+39.914854112" watchObservedRunningTime="2026-07-07 11:40:51.071878244 +0000 UTC m=+39.915602345" Jul 07 11:40:51.307009 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:51.306985 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rpxgg_55207ce8-d0ca-4d60-a966-daa4e9ecca81/dns-node-resolver/0.log" Jul 07 11:40:52.027126 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:52.027095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" event={"ID":"17b3e0f5-3744-471c-83a6-13588da0fdb8","Type":"ContainerStarted","Data":"11e39bd4bdd5d01158a777e3f1145a1780564872fafdc9ff2b1b8b64ac58d19d"} Jul 07 11:40:52.027714 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:52.027147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" event={"ID":"17b3e0f5-3744-471c-83a6-13588da0fdb8","Type":"ContainerStarted","Data":"e7562a085e08e3b383d1f8733f8b580953448a92717b17ab22b4a9eff8ee9580"} Jul 07 11:40:52.048009 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:52.047966 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-tlbfl" podStartSLOduration=2.007561289 podStartE2EDuration="4.04795245s" totalImagesPullingTime="2.040391161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:52.047036513 +0000 UTC m=+40.890760613" watchObservedRunningTime="2026-07-07 11:40:52.04795245 +0000 UTC m=+40.891676549" Jul 07 11:40:52.493959 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:52.493934 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ztk6q_b9d58c15-95ca-4172-a2ab-c76893c6ebea/node-ca/0.log" Jul 07 11:40:52.633789 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:52.633759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:52.633942 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:52.633923 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:40:52.634010 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:52.633998 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls podName:6583d603-c819-4182-a976-595b8fa41475 nodeName:}" failed. No retries permitted until 2026-07-07 11:40:56.633977799 +0000 UTC m=+45.477701896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlxns" (UID: "6583d603-c819-4182-a976-595b8fa41475") : secret "insights-runtime-extractor-tls" not found Jul 07 11:40:53.097508 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:53.097481 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wcrp5_4aac21db-70cd-4e10-b7ff-700f56b241ba/serve-healthcheck-canary/0.log" Jul 07 11:40:53.299134 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:53.299107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-tlbfl_17b3e0f5-3744-471c-83a6-13588da0fdb8/migrator/0.log" Jul 07 11:40:53.494359 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:53.494337 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-tlbfl_17b3e0f5-3744-471c-83a6-13588da0fdb8/graceful-termination/0.log" Jul 07 11:40:54.747225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:54.747167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:40:54.747585 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:54.747319 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 11:40:54.747585 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:54.747338 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7766875f79-zmv45: secret "image-registry-tls" not found Jul 07 11:40:54.747585 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:54.747391 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls podName:0f7a8dfd-113d-4e30-9a4a-927004d0e035 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:02.747376562 +0000 UTC m=+51.591100640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls") pod "image-registry-7766875f79-zmv45" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035") : secret "image-registry-tls" not found Jul 07 11:40:54.847970 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:54.847941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:40:54.848096 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:54.848082 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 11:40:54.848147 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:54.848137 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls podName:8ebddf9a-a072-4c3c-a78e-047ae767d096 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:02.848122409 +0000 UTC m=+51.691846489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls") pod "dns-default-kc454" (UID: "8ebddf9a-a072-4c3c-a78e-047ae767d096") : secret "dns-default-metrics-tls" not found Jul 07 11:40:56.658483 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:40:56.658446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:40:56.658852 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:56.658597 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 11:40:56.658852 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:40:56.658659 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls podName:6583d603-c819-4182-a976-595b8fa41475 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:04.658643333 +0000 UTC m=+53.502367434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlxns" (UID: "6583d603-c819-4182-a976-595b8fa41475") : secret "insights-runtime-extractor-tls" not found Jul 07 11:41:02.799429 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:02.799392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:02.803012 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:02.802985 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"image-registry-7766875f79-zmv45\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:02.900383 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:02.900355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:41:02.901856 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:02.901829 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:02.902526 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:02.902500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ebddf9a-a072-4c3c-a78e-047ae767d096-metrics-tls\") pod \"dns-default-kc454\" (UID: \"8ebddf9a-a072-4c3c-a78e-047ae767d096\") " pod="openshift-dns/dns-default-kc454" Jul 07 11:41:02.944583 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:02.944552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kc454" Jul 07 11:41:03.053347 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:03.053258 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7766875f79-zmv45"] Jul 07 11:41:03.057922 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:03.057897 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7a8dfd_113d_4e30_9a4a_927004d0e035.slice/crio-c2f8a6354d234fe4f841ff3482b6156e8745f1a11a28074ea0fd45a9008b39a4 WatchSource:0}: Error finding container c2f8a6354d234fe4f841ff3482b6156e8745f1a11a28074ea0fd45a9008b39a4: Status 404 returned error can't find the container with id c2f8a6354d234fe4f841ff3482b6156e8745f1a11a28074ea0fd45a9008b39a4 Jul 07 11:41:03.092744 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:03.092709 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kc454"] Jul 07 11:41:03.096663 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:03.096639 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ebddf9a_a072_4c3c_a78e_047ae767d096.slice/crio-ff040c41bdb484b65c16e941d1b8b7d84bdf5522da614ee00c774dff9ce189c4 WatchSource:0}: Error finding container ff040c41bdb484b65c16e941d1b8b7d84bdf5522da614ee00c774dff9ce189c4: Status 404 returned error can't find the container with id ff040c41bdb484b65c16e941d1b8b7d84bdf5522da614ee00c774dff9ce189c4 Jul 07 11:41:04.058474 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.058435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc454" event={"ID":"8ebddf9a-a072-4c3c-a78e-047ae767d096","Type":"ContainerStarted","Data":"ff040c41bdb484b65c16e941d1b8b7d84bdf5522da614ee00c774dff9ce189c4"} Jul 07 11:41:04.060142 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.060110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7766875f79-zmv45" event={"ID":"0f7a8dfd-113d-4e30-9a4a-927004d0e035","Type":"ContainerStarted","Data":"04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab"} Jul 07 11:41:04.060313 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.060161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7766875f79-zmv45" event={"ID":"0f7a8dfd-113d-4e30-9a4a-927004d0e035","Type":"ContainerStarted","Data":"c2f8a6354d234fe4f841ff3482b6156e8745f1a11a28074ea0fd45a9008b39a4"} Jul 07 11:41:04.060313 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.060248 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:04.082998 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.082959 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7766875f79-zmv45" podStartSLOduration=24.082943456 podStartE2EDuration="24.082943456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:04.082088875 +0000 UTC m=+52.925812990" watchObservedRunningTime="2026-07-07 11:41:04.082943456 +0000 UTC m=+52.926667555" Jul 07 11:41:04.713081 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.713046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:41:04.715278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.715248 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6583d603-c819-4182-a976-595b8fa41475-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlxns\" (UID: \"6583d603-c819-4182-a976-595b8fa41475\") " pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:41:04.792390 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:04.792367 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vlxns" Jul 07 11:41:05.094270 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:05.094237 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vlxns"] Jul 07 11:41:05.097448 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:05.097428 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6583d603_c819_4182_a976_595b8fa41475.slice/crio-aefbd3e7ee6233767e7b4b5c1af0d5f4237db3a3439c466fe20ce3f928b4a4fa WatchSource:0}: Error finding container aefbd3e7ee6233767e7b4b5c1af0d5f4237db3a3439c466fe20ce3f928b4a4fa: Status 404 returned error can't find the container with id aefbd3e7ee6233767e7b4b5c1af0d5f4237db3a3439c466fe20ce3f928b4a4fa Jul 07 11:41:06.068552 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:06.068512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlxns" event={"ID":"6583d603-c819-4182-a976-595b8fa41475","Type":"ContainerStarted","Data":"aefbd3e7ee6233767e7b4b5c1af0d5f4237db3a3439c466fe20ce3f928b4a4fa"} Jul 07 11:41:06.071096 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:06.071060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc454" event={"ID":"8ebddf9a-a072-4c3c-a78e-047ae767d096","Type":"ContainerStarted","Data":"5db40b5f0fea385330a7cb9d134fa03c6ea7d1f7a5c2065b50cc1dc07ed5f743"} Jul 07 11:41:06.071237 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:06.071104 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc454" event={"ID":"8ebddf9a-a072-4c3c-a78e-047ae767d096","Type":"ContainerStarted","Data":"a66c4e384b15144e330f4430be5b2f810f6e82a11c5cf7f122400347caac8433"} Jul 07 11:41:06.071305 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:06.071235 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kc454" Jul 07 11:41:06.089908 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:06.089859 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kc454" podStartSLOduration=18.215777007 podStartE2EDuration="20.089841316s" totalImagesPullingTime="1.874064309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:06.089275536 +0000 UTC m=+54.932999640" watchObservedRunningTime="2026-07-07 11:41:06.089841316 +0000 UTC m=+54.933565416" Jul 07 11:41:09.083396 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:09.083316 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlxns" event={"ID":"6583d603-c819-4182-a976-595b8fa41475","Type":"ContainerStarted","Data":"acc27642a7b4362555823e164a5a325cd6202e25ef399370efa7b0c9742e7d85"} Jul 07 11:41:10.088927 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:10.088878 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlxns" event={"ID":"6583d603-c819-4182-a976-595b8fa41475","Type":"ContainerStarted","Data":"20f8e1a242c347a421a694ef1ff32a02223afb9ca552a68fa7216199f78ad230"} Jul 07 11:41:12.764010 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.763922 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7766875f79-zmv45"] Jul 07 11:41:12.815656 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.815626 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb"] Jul 07 11:41:12.840564 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.840530 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb"] Jul 07 11:41:12.840694 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.840611 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:12.847115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.847092 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 11:41:12.847345 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.847328 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 07 11:41:12.847463 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.847366 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-dnbsx\"" Jul 07 11:41:12.847463 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.847399 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 11:41:12.847557 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.847465 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 11:41:12.876594 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.876565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a8471b90-ba1f-41aa-82b3-1f0687229ee6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66c79dfc96-prfwb\" (UID: \"a8471b90-ba1f-41aa-82b3-1f0687229ee6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:12.876706 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.876600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvc8\" (UniqueName: \"kubernetes.io/projected/a8471b90-ba1f-41aa-82b3-1f0687229ee6-kube-api-access-bvvc8\") pod \"managed-serviceaccount-addon-agent-66c79dfc96-prfwb\" (UID: \"a8471b90-ba1f-41aa-82b3-1f0687229ee6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:12.913231 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.913166 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7759c787bb-wkb9p"] Jul 07 11:41:12.935129 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.935109 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.936000 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.935982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7759c787bb-wkb9p"] Jul 07 11:41:12.977150 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe1dc1c-d412-4e29-8a50-75e632665d6a-trusted-ca\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977350 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f99k4\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-kube-api-access-f99k4\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977350 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe1dc1c-d412-4e29-8a50-75e632665d6a-image-registry-private-configuration\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977350 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe1dc1c-d412-4e29-8a50-75e632665d6a-ca-trust-extracted\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977350 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-bound-sa-token\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977566 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe1dc1c-d412-4e29-8a50-75e632665d6a-registry-certificates\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977566 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a8471b90-ba1f-41aa-82b3-1f0687229ee6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66c79dfc96-prfwb\" (UID: \"a8471b90-ba1f-41aa-82b3-1f0687229ee6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:12.977566 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvc8\" (UniqueName: \"kubernetes.io/projected/a8471b90-ba1f-41aa-82b3-1f0687229ee6-kube-api-access-bvvc8\") pod \"managed-serviceaccount-addon-agent-66c79dfc96-prfwb\" (UID: \"a8471b90-ba1f-41aa-82b3-1f0687229ee6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:12.977566 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe1dc1c-d412-4e29-8a50-75e632665d6a-installation-pull-secrets\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.977566 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.977556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-registry-tls\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:12.979878 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.979858 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a8471b90-ba1f-41aa-82b3-1f0687229ee6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66c79dfc96-prfwb\" (UID: \"a8471b90-ba1f-41aa-82b3-1f0687229ee6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:12.987130 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:12.987102 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvc8\" (UniqueName: \"kubernetes.io/projected/a8471b90-ba1f-41aa-82b3-1f0687229ee6-kube-api-access-bvvc8\") pod \"managed-serviceaccount-addon-agent-66c79dfc96-prfwb\" (UID: \"a8471b90-ba1f-41aa-82b3-1f0687229ee6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:13.002407 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.002386 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c7qq" Jul 07 11:41:13.078425 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe1dc1c-d412-4e29-8a50-75e632665d6a-trusted-ca\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078425 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f99k4\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-kube-api-access-f99k4\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078425 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe1dc1c-d412-4e29-8a50-75e632665d6a-image-registry-private-configuration\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078770 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe1dc1c-d412-4e29-8a50-75e632665d6a-ca-trust-extracted\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078770 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-bound-sa-token\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078770 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe1dc1c-d412-4e29-8a50-75e632665d6a-registry-certificates\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078985 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078857 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6fe1dc1c-d412-4e29-8a50-75e632665d6a-ca-trust-extracted\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.078985 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.078951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe1dc1c-d412-4e29-8a50-75e632665d6a-installation-pull-secrets\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.079093 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.079011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-registry-tls\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.079574 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.079550 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6fe1dc1c-d412-4e29-8a50-75e632665d6a-trusted-ca\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.079712 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.079629 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6fe1dc1c-d412-4e29-8a50-75e632665d6a-registry-certificates\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.080860 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.080841 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6fe1dc1c-d412-4e29-8a50-75e632665d6a-image-registry-private-configuration\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.081256 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.081240 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-registry-tls\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.081360 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.081345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6fe1dc1c-d412-4e29-8a50-75e632665d6a-installation-pull-secrets\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.091239 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.091215 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f99k4\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-kube-api-access-f99k4\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.093419 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.093398 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fe1dc1c-d412-4e29-8a50-75e632665d6a-bound-sa-token\") pod \"image-registry-7759c787bb-wkb9p\" (UID: \"6fe1dc1c-d412-4e29-8a50-75e632665d6a\") " pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.099651 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.099631 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlxns" event={"ID":"6583d603-c819-4182-a976-595b8fa41475","Type":"ContainerStarted","Data":"fde428bc044c8de52af430659a30d3c711e634d4a7bfdbcad217ff387c21f8f4"} Jul 07 11:41:13.164683 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.164649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" Jul 07 11:41:13.244548 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.244516 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:13.314842 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.314793 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vlxns" podStartSLOduration=18.055274542 podStartE2EDuration="25.314775666s" totalImagesPullingTime="7.259501124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:13.121345745 +0000 UTC m=+61.965069855" watchObservedRunningTime="2026-07-07 11:41:13.314775666 +0000 UTC m=+62.158499765" Jul 07 11:41:13.314973 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.314929 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb"] Jul 07 11:41:13.317899 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:13.317875 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8471b90_ba1f_41aa_82b3_1f0687229ee6.slice/crio-c936f22a4ce998c3ae2aa7dc13a925b7ea80dfec3c82cea255954a8e63dc2b9f WatchSource:0}: Error finding container c936f22a4ce998c3ae2aa7dc13a925b7ea80dfec3c82cea255954a8e63dc2b9f: Status 404 returned error can't find the container with id c936f22a4ce998c3ae2aa7dc13a925b7ea80dfec3c82cea255954a8e63dc2b9f Jul 07 11:41:13.375608 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:13.375582 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7759c787bb-wkb9p"] Jul 07 11:41:13.378489 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:13.378466 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe1dc1c_d412_4e29_8a50_75e632665d6a.slice/crio-2ec500e86c5141e6be4de5a0c17319544c23575ff65d749c337f57461a06200f WatchSource:0}: Error finding container 2ec500e86c5141e6be4de5a0c17319544c23575ff65d749c337f57461a06200f: Status 404 returned error can't find the container with id 2ec500e86c5141e6be4de5a0c17319544c23575ff65d749c337f57461a06200f Jul 07 11:41:14.104990 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:14.104904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" event={"ID":"6fe1dc1c-d412-4e29-8a50-75e632665d6a","Type":"ContainerStarted","Data":"2cfc3287091b942c47a563e7a169cddeee259da3253fbbd786c237c54391ed05"} Jul 07 11:41:14.104990 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:14.104962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" event={"ID":"6fe1dc1c-d412-4e29-8a50-75e632665d6a","Type":"ContainerStarted","Data":"2ec500e86c5141e6be4de5a0c17319544c23575ff65d749c337f57461a06200f"} Jul 07 11:41:14.105524 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:14.105105 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:14.107280 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:14.107238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" event={"ID":"a8471b90-ba1f-41aa-82b3-1f0687229ee6","Type":"ContainerStarted","Data":"c936f22a4ce998c3ae2aa7dc13a925b7ea80dfec3c82cea255954a8e63dc2b9f"} Jul 07 11:41:14.135869 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:14.135807 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" podStartSLOduration=2.135788826 podStartE2EDuration="2.135788826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:14.133565575 +0000 UTC m=+62.977289676" watchObservedRunningTime="2026-07-07 11:41:14.135788826 +0000 UTC m=+62.979512926" Jul 07 11:41:16.078036 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:16.077955 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kc454" Jul 07 11:41:16.115690 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:16.115661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" event={"ID":"a8471b90-ba1f-41aa-82b3-1f0687229ee6","Type":"ContainerStarted","Data":"83e383882d0cfceaf94e2c0c033987a879422130e910f91943bfd1e35bec91e4"} Jul 07 11:41:16.134141 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:16.134095 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66c79dfc96-prfwb" podStartSLOduration=1.8031945820000002 podStartE2EDuration="4.134080895s" totalImagesPullingTime="2.330886313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:16.133185082 +0000 UTC m=+64.976909181" watchObservedRunningTime="2026-07-07 11:41:16.134080895 +0000 UTC m=+64.977804995" Jul 07 11:41:17.415755 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.415722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:41:17.418277 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.418260 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:41:17.428146 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.428116 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/529387f7-20c3-4ce0-aa48-3861845232aa-metrics-certs\") pod \"network-metrics-daemon-2ml8h\" (UID: \"529387f7-20c3-4ce0-aa48-3861845232aa\") " pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:41:17.516463 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.516434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:41:17.516601 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.516473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:41:17.518787 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.518770 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:41:17.518848 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.518807 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:41:17.529694 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.529674 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a31e3c1-5122-433a-88e1-da067af93778-original-pull-secret\") pod \"global-pull-secret-syncer-v7v7r\" (UID: \"1a31e3c1-5122-433a-88e1-da067af93778\") " pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:41:17.530013 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.530001 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:41:17.540000 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.539981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6sx\" (UniqueName: \"kubernetes.io/projected/f0a98300-a855-4d77-a89c-100c5e06e501-kube-api-access-9m6sx\") pod \"network-check-target-qdr6z\" (UID: \"f0a98300-a855-4d77-a89c-100c5e06e501\") " pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:41:17.545869 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.545851 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v7v7r" Jul 07 11:41:17.555174 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.555157 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vqglz\"" Jul 07 11:41:17.562873 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.562857 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:41:17.642221 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.642177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cprtm\"" Jul 07 11:41:17.649469 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.649443 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2ml8h" Jul 07 11:41:17.674749 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.674616 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v7v7r"] Jul 07 11:41:17.677281 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:17.677250 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a31e3c1_5122_433a_88e1_da067af93778.slice/crio-1b28deb836b394c27c2e7eac150b81f023c4bab719825507febf5304711c7def WatchSource:0}: Error finding container 1b28deb836b394c27c2e7eac150b81f023c4bab719825507febf5304711c7def: Status 404 returned error can't find the container with id 1b28deb836b394c27c2e7eac150b81f023c4bab719825507febf5304711c7def Jul 07 11:41:17.692562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.692538 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qdr6z"] Jul 07 11:41:17.695924 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:17.695887 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a98300_a855_4d77_a89c_100c5e06e501.slice/crio-e355d7b9971acf18602ddc838c6e70c9abdeb01263f9f2364eccb3ea5efab447 WatchSource:0}: Error finding container e355d7b9971acf18602ddc838c6e70c9abdeb01263f9f2364eccb3ea5efab447: Status 404 returned error can't find the container with id e355d7b9971acf18602ddc838c6e70c9abdeb01263f9f2364eccb3ea5efab447 Jul 07 11:41:17.781752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:17.781716 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2ml8h"] Jul 07 11:41:17.784250 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:17.784221 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529387f7_20c3_4ce0_aa48_3861845232aa.slice/crio-f12b9c6a596f571702cc11a34522250e2315170cef7b524d6c1d9d41082e3181 WatchSource:0}: Error finding container f12b9c6a596f571702cc11a34522250e2315170cef7b524d6c1d9d41082e3181: Status 404 returned error can't find the container with id f12b9c6a596f571702cc11a34522250e2315170cef7b524d6c1d9d41082e3181 Jul 07 11:41:18.123994 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:18.123956 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qdr6z" event={"ID":"f0a98300-a855-4d77-a89c-100c5e06e501","Type":"ContainerStarted","Data":"e355d7b9971acf18602ddc838c6e70c9abdeb01263f9f2364eccb3ea5efab447"} Jul 07 11:41:18.125337 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:18.125309 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v7v7r" event={"ID":"1a31e3c1-5122-433a-88e1-da067af93778","Type":"ContainerStarted","Data":"1b28deb836b394c27c2e7eac150b81f023c4bab719825507febf5304711c7def"} Jul 07 11:41:18.126579 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:18.126549 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2ml8h" event={"ID":"529387f7-20c3-4ce0-aa48-3861845232aa","Type":"ContainerStarted","Data":"f12b9c6a596f571702cc11a34522250e2315170cef7b524d6c1d9d41082e3181"} Jul 07 11:41:19.785004 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.784923 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x"] Jul 07 11:41:19.803956 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.803880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:19.808227 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.808189 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 11:41:19.809332 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.809161 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vcln5\"" Jul 07 11:41:19.809332 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.809186 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 11:41:19.809332 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.809273 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 11:41:19.809332 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.809185 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 11:41:19.809574 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.809444 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 11:41:19.816651 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.816628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x"] Jul 07 11:41:19.932968 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.932935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thxqm\" (UniqueName: \"kubernetes.io/projected/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-kube-api-access-thxqm\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:19.933165 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.932979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:19.933165 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.933009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:19.933165 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:19.933060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.034423 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.034392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thxqm\" (UniqueName: \"kubernetes.io/projected/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-kube-api-access-thxqm\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.034660 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.034431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.034660 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.034467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.034660 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.034506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.035594 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.035534 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.037798 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.037771 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.037953 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.037928 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.047804 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.047778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thxqm\" (UniqueName: \"kubernetes.io/projected/91c6f39d-d6b0-4449-8857-46ed8bf3c45b-kube-api-access-thxqm\") pod \"prometheus-operator-567f7b8b7c-6hh2x\" (UID: \"91c6f39d-d6b0-4449-8857-46ed8bf3c45b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.116272 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.116234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" Jul 07 11:41:20.137424 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.137392 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2ml8h" event={"ID":"529387f7-20c3-4ce0-aa48-3861845232aa","Type":"ContainerStarted","Data":"35a82a2aaaa1f9ea50ce00a86319db9d22d2c939858b11a24c4dbbb79a6dcc08"} Jul 07 11:41:20.137598 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.137450 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2ml8h" event={"ID":"529387f7-20c3-4ce0-aa48-3861845232aa","Type":"ContainerStarted","Data":"16e6d63a46b1ce5bbd232dee59b8310a1d61e2a2fa6ead26b1669c3e36f9c85c"} Jul 07 11:41:20.165660 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:20.165613 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2ml8h" podStartSLOduration=66.708994117 podStartE2EDuration="1m8.165594114s" totalImagesPullingTime="1.456599997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:20.164828915 +0000 UTC m=+69.008553016" watchObservedRunningTime="2026-07-07 11:41:20.165594114 +0000 UTC m=+69.009318240" Jul 07 11:41:22.333452 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:22.333345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x"] Jul 07 11:41:22.336780 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:22.336752 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c6f39d_d6b0_4449_8857_46ed8bf3c45b.slice/crio-97d1018a6d822490c0706ab98d0b0cc13a377e42801a95682ed2b25e34f83121 WatchSource:0}: Error finding container 97d1018a6d822490c0706ab98d0b0cc13a377e42801a95682ed2b25e34f83121: Status 404 returned error can't find the container with id 97d1018a6d822490c0706ab98d0b0cc13a377e42801a95682ed2b25e34f83121 Jul 07 11:41:22.770467 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:22.770428 2580 patch_prober.go:28] interesting pod/image-registry-7766875f79-zmv45 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 11:41:22.770648 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:22.770491 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7766875f79-zmv45" podUID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 11:41:23.153457 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:23.153414 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qdr6z" event={"ID":"f0a98300-a855-4d77-a89c-100c5e06e501","Type":"ContainerStarted","Data":"aad75facac2599cdb303afb51646ccea36ab34b383c9d3bf66c0ee26bda9ba2e"} Jul 07 11:41:23.153636 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:23.153544 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:41:23.155138 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:23.155111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v7v7r" event={"ID":"1a31e3c1-5122-433a-88e1-da067af93778","Type":"ContainerStarted","Data":"3838c27861d489c4146cde232f7810a6a4c623d94525bf5a58f1c6e21337a7cd"} Jul 07 11:41:23.156433 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:23.156405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" event={"ID":"91c6f39d-d6b0-4449-8857-46ed8bf3c45b","Type":"ContainerStarted","Data":"97d1018a6d822490c0706ab98d0b0cc13a377e42801a95682ed2b25e34f83121"} Jul 07 11:41:23.184115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:23.184068 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qdr6z" podStartSLOduration=66.674467287 podStartE2EDuration="1m11.184054342s" totalImagesPullingTime="4.509587055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:23.18321035 +0000 UTC m=+72.026934442" watchObservedRunningTime="2026-07-07 11:41:23.184054342 +0000 UTC m=+72.027778442" Jul 07 11:41:23.207361 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:23.207316 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-v7v7r" podStartSLOduration=66.678627062 podStartE2EDuration="1m11.207302482s" totalImagesPullingTime="4.52867542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:23.207227006 +0000 UTC m=+72.050951106" watchObservedRunningTime="2026-07-07 11:41:23.207302482 +0000 UTC m=+72.051026581" Jul 07 11:41:24.161439 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.161406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" event={"ID":"91c6f39d-d6b0-4449-8857-46ed8bf3c45b","Type":"ContainerStarted","Data":"8c61accfa0363bc16c19db0f44c941e2dcf21e6c7ce94dc3060f67996ca88c1b"} Jul 07 11:41:24.161816 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.161459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" event={"ID":"91c6f39d-d6b0-4449-8857-46ed8bf3c45b","Type":"ContainerStarted","Data":"08fde69a467c7ffe74fb80ce5444cbdc663ad17fff924a3e57523fb372c111f2"} Jul 07 11:41:24.185069 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.185014 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-6hh2x" podStartSLOduration=4.117359615 podStartE2EDuration="5.184998985s" totalImagesPullingTime="1.06763937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:24.183972475 +0000 UTC m=+73.027696574" watchObservedRunningTime="2026-07-07 11:41:24.184998985 +0000 UTC m=+73.028723085" Jul 07 11:41:24.428295 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.428211 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-tdg97"] Jul 07 11:41:24.430026 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.430012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:24.432134 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.432109 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n6rtq\"" Jul 07 11:41:24.432134 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.432120 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 11:41:24.432564 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.432548 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 11:41:24.442846 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.442824 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-tdg97"] Jul 07 11:41:24.472004 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.471975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hx4j\" (UniqueName: \"kubernetes.io/projected/de8b8d4e-3667-4306-80d3-66b26df43a1b-kube-api-access-6hx4j\") pod \"downloads-685bfb5f64-tdg97\" (UID: \"de8b8d4e-3667-4306-80d3-66b26df43a1b\") " pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:24.572361 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.572328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hx4j\" (UniqueName: \"kubernetes.io/projected/de8b8d4e-3667-4306-80d3-66b26df43a1b-kube-api-access-6hx4j\") pod \"downloads-685bfb5f64-tdg97\" (UID: \"de8b8d4e-3667-4306-80d3-66b26df43a1b\") " pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:24.585706 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.585681 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hx4j\" (UniqueName: \"kubernetes.io/projected/de8b8d4e-3667-4306-80d3-66b26df43a1b-kube-api-access-6hx4j\") pod \"downloads-685bfb5f64-tdg97\" (UID: \"de8b8d4e-3667-4306-80d3-66b26df43a1b\") " pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:24.738572 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.738489 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:24.867795 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:24.867767 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-tdg97"] Jul 07 11:41:24.870979 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:24.870952 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8b8d4e_3667_4306_80d3_66b26df43a1b.slice/crio-54e4c994a5243a8b27af10c0f99f91e63031271a537aa35badc70c54e1fc527c WatchSource:0}: Error finding container 54e4c994a5243a8b27af10c0f99f91e63031271a537aa35badc70c54e1fc527c: Status 404 returned error can't find the container with id 54e4c994a5243a8b27af10c0f99f91e63031271a537aa35badc70c54e1fc527c Jul 07 11:41:25.165878 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:25.165844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-tdg97" event={"ID":"de8b8d4e-3667-4306-80d3-66b26df43a1b","Type":"ContainerStarted","Data":"54e4c994a5243a8b27af10c0f99f91e63031271a537aa35badc70c54e1fc527c"} Jul 07 11:41:26.288771 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.288741 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nrbp2"] Jul 07 11:41:26.291450 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.291423 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.292354 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.292305 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs"] Jul 07 11:41:26.294566 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.294543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.296662 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.296640 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 11:41:26.296753 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.296702 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 11:41:26.296880 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.296850 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 11:41:26.301876 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.301717 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 11:41:26.302082 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.302029 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 11:41:26.302376 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.302358 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cl6bx\"" Jul 07 11:41:26.302507 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.302493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cfrqc\"" Jul 07 11:41:26.310930 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.310910 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 11:41:26.344084 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.343968 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs"] Jul 07 11:41:26.387792 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.387935 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-sys\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.387935 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387844 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.387935 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.387935 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-wtmp\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6glt\" (UniqueName: \"kubernetes.io/projected/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-api-access-x6glt\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.388136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.387974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-tls\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.388136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-root\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.388136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llrvz\" (UniqueName: \"kubernetes.io/projected/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-kube-api-access-llrvz\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388395 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-metrics-client-ca\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388395 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388395 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-textfile\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.388395 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.388240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.489452 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6glt\" (UniqueName: \"kubernetes.io/projected/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-api-access-x6glt\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-tls\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-root\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llrvz\" (UniqueName: \"kubernetes.io/projected/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-kube-api-access-llrvz\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-metrics-client-ca\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.489640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489633 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-root\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-textfile\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-sys\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489836 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-wtmp\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.489947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-sys\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490115 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.490073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-wtmp\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490635 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.490277 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.490635 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.490480 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-textfile\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490746 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.490668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.490746 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.490714 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.490985 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.490962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-metrics-client-ca\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.491230 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.491176 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.492501 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.492477 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-tls\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.492602 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.492534 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.492723 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.492701 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.493041 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.492999 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.500483 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.500440 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6glt\" (UniqueName: \"kubernetes.io/projected/eb8ca7c0-60ef-453f-81c7-77dfe9000b45-kube-api-access-x6glt\") pod \"kube-state-metrics-7c9dbb8b7d-8dbvs\" (UID: \"eb8ca7c0-60ef-453f-81c7-77dfe9000b45\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.507346 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.507328 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llrvz\" (UniqueName: \"kubernetes.io/projected/3cadb9d6-22ba-49c5-9aaf-95752c1a25a8-kube-api-access-llrvz\") pod \"node-exporter-nrbp2\" (UID: \"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8\") " pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.604937 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.604867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nrbp2" Jul 07 11:41:26.611590 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.611567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" Jul 07 11:41:26.615813 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:26.615787 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cadb9d6_22ba_49c5_9aaf_95752c1a25a8.slice/crio-fd8b138ade6ebfff8c709f599490d505cf973683882300ce912ba03ec1c0c214 WatchSource:0}: Error finding container fd8b138ade6ebfff8c709f599490d505cf973683882300ce912ba03ec1c0c214: Status 404 returned error can't find the container with id fd8b138ade6ebfff8c709f599490d505cf973683882300ce912ba03ec1c0c214 Jul 07 11:41:26.779779 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:26.779741 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8ca7c0_60ef_453f_81c7_77dfe9000b45.slice/crio-aab750ba9f5ccfa81a978bc0543d459e1604f53f00697b5757e8fb45bbb90360 WatchSource:0}: Error finding container aab750ba9f5ccfa81a978bc0543d459e1604f53f00697b5757e8fb45bbb90360: Status 404 returned error can't find the container with id aab750ba9f5ccfa81a978bc0543d459e1604f53f00697b5757e8fb45bbb90360 Jul 07 11:41:26.781343 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:26.781316 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs"] Jul 07 11:41:27.175745 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:27.175702 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrbp2" event={"ID":"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8","Type":"ContainerStarted","Data":"fd8b138ade6ebfff8c709f599490d505cf973683882300ce912ba03ec1c0c214"} Jul 07 11:41:27.177152 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:27.177120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" event={"ID":"eb8ca7c0-60ef-453f-81c7-77dfe9000b45","Type":"ContainerStarted","Data":"aab750ba9f5ccfa81a978bc0543d459e1604f53f00697b5757e8fb45bbb90360"} Jul 07 11:41:28.185529 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:28.184710 2580 generic.go:356] "Generic (PLEG): container finished" podID="3cadb9d6-22ba-49c5-9aaf-95752c1a25a8" containerID="b2fd032fab6635f4c366248a198a5cde6d4ff2103ce370e26b15e84e39de2dbc" exitCode=0 Jul 07 11:41:28.185529 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:28.184821 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrbp2" event={"ID":"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8","Type":"ContainerDied","Data":"b2fd032fab6635f4c366248a198a5cde6d4ff2103ce370e26b15e84e39de2dbc"} Jul 07 11:41:28.188218 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:28.188109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" event={"ID":"eb8ca7c0-60ef-453f-81c7-77dfe9000b45","Type":"ContainerStarted","Data":"446ba14cbc7ffa21bbea605cda9c5ac33aa7ae58fbbc69662ce2a7ccc1c269b1"} Jul 07 11:41:29.195769 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:29.195726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrbp2" event={"ID":"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8","Type":"ContainerStarted","Data":"62cc74e8e40cefac9454beef477ca585d7d6c7463e152589750e234b822fb9c9"} Jul 07 11:41:29.196249 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:29.195799 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrbp2" event={"ID":"3cadb9d6-22ba-49c5-9aaf-95752c1a25a8","Type":"ContainerStarted","Data":"657673dcab9b3b726ea2e95b1ac2d5ab3c9f3c82932283871ab834fd57c55bc0"} Jul 07 11:41:29.198656 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:29.198623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" event={"ID":"eb8ca7c0-60ef-453f-81c7-77dfe9000b45","Type":"ContainerStarted","Data":"36c985319f4a4cde5fe9f2cd6cd7f0f267269e24a8755735ef5c6b2a64c0b2b1"} Jul 07 11:41:29.198788 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:29.198687 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" event={"ID":"eb8ca7c0-60ef-453f-81c7-77dfe9000b45","Type":"ContainerStarted","Data":"0baa6f73341248caaf0fc678c940ff9ad4e20474543c086abdf07e7ac4559cb6"} Jul 07 11:41:29.234052 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:29.233995 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nrbp2" podStartSLOduration=2.169830049 podStartE2EDuration="3.233978484s" totalImagesPullingTime="989.919585ms" totalInitContainerRuntime="74.22885ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:29.23335818 +0000 UTC m=+78.077082294" watchObservedRunningTime="2026-07-07 11:41:29.233978484 +0000 UTC m=+78.077702565" Jul 07 11:41:29.274140 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:29.274094 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8dbvs" podStartSLOduration=1.9967282530000001 podStartE2EDuration="3.274079451s" totalImagesPullingTime="1.277351198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:29.272406654 +0000 UTC m=+78.116130765" watchObservedRunningTime="2026-07-07 11:41:29.274079451 +0000 UTC m=+78.117803551" Jul 07 11:41:31.003898 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.003401 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z"] Jul 07 11:41:31.006221 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.006166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.008828 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.008803 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 11:41:31.009901 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.009876 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fqhhl\"" Jul 07 11:41:31.017093 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.017072 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z"] Jul 07 11:41:31.134924 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.134887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwhw7\" (UniqueName: \"kubernetes.io/projected/624086bc-f466-4257-b73b-d514b06fd259-kube-api-access-lwhw7\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.135073 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.134952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624086bc-f466-4257-b73b-d514b06fd259-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.236153 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.236120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwhw7\" (UniqueName: \"kubernetes.io/projected/624086bc-f466-4257-b73b-d514b06fd259-kube-api-access-lwhw7\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.236345 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.236179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624086bc-f466-4257-b73b-d514b06fd259-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.236345 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:41:31.236318 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 11:41:31.236633 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:41:31.236382 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624086bc-f466-4257-b73b-d514b06fd259-monitoring-plugin-cert podName:624086bc-f466-4257-b73b-d514b06fd259 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:31.736359325 +0000 UTC m=+80.580083441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/624086bc-f466-4257-b73b-d514b06fd259-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-g8t4z" (UID: "624086bc-f466-4257-b73b-d514b06fd259") : secret "monitoring-plugin-cert" not found Jul 07 11:41:31.246381 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.246354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwhw7\" (UniqueName: \"kubernetes.io/projected/624086bc-f466-4257-b73b-d514b06fd259-kube-api-access-lwhw7\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.740511 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.740472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624086bc-f466-4257-b73b-d514b06fd259-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.743133 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.743107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624086bc-f466-4257-b73b-d514b06fd259-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-g8t4z\" (UID: \"624086bc-f466-4257-b73b-d514b06fd259\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:31.918182 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:31.918157 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:32.055630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.055599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z"] Jul 07 11:41:32.059213 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:32.059165 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624086bc_f466_4257_b73b_d514b06fd259.slice/crio-9f72685e5e6bb30fe6c3b30b773d828a82d89970b0d67d9216944a33ad80b5f5 WatchSource:0}: Error finding container 9f72685e5e6bb30fe6c3b30b773d828a82d89970b0d67d9216944a33ad80b5f5: Status 404 returned error can't find the container with id 9f72685e5e6bb30fe6c3b30b773d828a82d89970b0d67d9216944a33ad80b5f5 Jul 07 11:41:32.215075 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.215043 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" event={"ID":"624086bc-f466-4257-b73b-d514b06fd259","Type":"ContainerStarted","Data":"9f72685e5e6bb30fe6c3b30b773d828a82d89970b0d67d9216944a33ad80b5f5"} Jul 07 11:41:32.724234 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.724188 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:41:32.728512 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.728489 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.733951 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.733902 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:41:32.734078 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.733987 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734111 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734248 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734261 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734376 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734756 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734772 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sbv7b\"" Jul 07 11:41:32.735056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.734953 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:41:32.735490 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.735261 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:41:32.735490 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.735281 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:41:32.735591 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.735501 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:41:32.735591 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.735522 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:41:32.736630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.736462 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-29jv15socv5rf\"" Jul 07 11:41:32.738327 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.738004 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:41:32.746381 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.746361 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:41:32.755666 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.755557 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:41:32.769264 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.769239 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:32.848953 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.848908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.848953 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.848956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-config\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.848981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849014 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849225 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849367 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfd2s\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-kube-api-access-zfd2s\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849497 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849938 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.849938 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.849581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.950947 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.950908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.950947 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.950951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-config\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.950974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfd2s\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-kube-api-access-zfd2s\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.951581 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.952043 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951733 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.952043 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.951795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.953279 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.952965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.955161 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.955134 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.955348 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.955326 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.955704 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.955618 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-config\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.956216 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.955948 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.956317 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.956234 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.960177 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.956489 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.960177 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.957049 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.960177 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.957267 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.960177 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.960075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.960632 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.960612 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.961346 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.960776 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.963265 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.961822 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.963265 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.962078 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.963265 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.962169 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.963265 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.962215 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.963515 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.963496 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:32.975921 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:32.975867 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfd2s\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-kube-api-access-zfd2s\") pod \"prometheus-k8s-0\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:33.044744 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:33.044614 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:33.451177 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:33.451122 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:41:33.456291 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:41:33.456264 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac05307f_6e1c_43ea_a139_61f19d0374f6.slice/crio-d925c40d255e177b6a0b29c0db660e786585b888fb8f5f881b5619d4f2bdfdc2 WatchSource:0}: Error finding container d925c40d255e177b6a0b29c0db660e786585b888fb8f5f881b5619d4f2bdfdc2: Status 404 returned error can't find the container with id d925c40d255e177b6a0b29c0db660e786585b888fb8f5f881b5619d4f2bdfdc2 Jul 07 11:41:34.224910 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:34.224875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" event={"ID":"624086bc-f466-4257-b73b-d514b06fd259","Type":"ContainerStarted","Data":"eb81e55bad940635b0467cea35783a9b3f9970e239ba67a0e1ad67091a3b8d65"} Jul 07 11:41:34.225113 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:34.224938 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:34.227013 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:34.226976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"d925c40d255e177b6a0b29c0db660e786585b888fb8f5f881b5619d4f2bdfdc2"} Jul 07 11:41:34.230892 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:34.230871 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" Jul 07 11:41:34.245278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:34.245241 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-g8t4z" podStartSLOduration=2.952646052 podStartE2EDuration="4.245228014s" totalImagesPullingTime="1.292581962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:34.244366728 +0000 UTC m=+83.088090830" watchObservedRunningTime="2026-07-07 11:41:34.245228014 +0000 UTC m=+83.088952114" Jul 07 11:41:35.115651 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:35.115623 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7759c787bb-wkb9p" Jul 07 11:41:35.232745 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:35.232712 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" exitCode=0 Jul 07 11:41:35.232897 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:35.232809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} Jul 07 11:41:37.784358 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:37.784311 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7766875f79-zmv45" podUID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" containerName="registry" containerID="cri-o://04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab" gracePeriod=30 Jul 07 11:41:42.704696 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.704672 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:42.843907 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.843838 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-bound-sa-token\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.843907 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.843873 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-certificates\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.843907 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.843898 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-trusted-ca\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.844184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.843941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f7a8dfd-113d-4e30-9a4a-927004d0e035-ca-trust-extracted\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.844184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.843997 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf85b\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-kube-api-access-mf85b\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.844184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844030 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-installation-pull-secrets\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.844184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844059 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.844184 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844095 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-image-registry-private-configuration\") pod \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\" (UID: \"0f7a8dfd-113d-4e30-9a4a-927004d0e035\") " Jul 07 11:41:42.844448 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844356 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:41:42.844448 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844382 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:41:42.844723 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844702 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-certificates\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.844808 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.844726 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f7a8dfd-113d-4e30-9a4a-927004d0e035-trusted-ca\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.846645 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.846619 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:41:42.846760 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.846657 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:41:42.846760 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.846732 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:41:42.846871 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.846828 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:41:42.846871 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.846828 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-kube-api-access-mf85b" (OuterVolumeSpecName: "kube-api-access-mf85b") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "kube-api-access-mf85b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:41:42.854956 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.854930 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f7a8dfd-113d-4e30-9a4a-927004d0e035-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0f7a8dfd-113d-4e30-9a4a-927004d0e035" (UID: "0f7a8dfd-113d-4e30-9a4a-927004d0e035"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:41:42.945325 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.945296 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf85b\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-kube-api-access-mf85b\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.945325 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.945323 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-installation-pull-secrets\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.945473 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.945338 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-registry-tls\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.945473 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.945354 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f7a8dfd-113d-4e30-9a4a-927004d0e035-image-registry-private-configuration\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.945473 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.945369 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f7a8dfd-113d-4e30-9a4a-927004d0e035-bound-sa-token\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:42.945473 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:42.945382 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f7a8dfd-113d-4e30-9a4a-927004d0e035-ca-trust-extracted\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:41:43.269605 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.269568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-tdg97" event={"ID":"de8b8d4e-3667-4306-80d3-66b26df43a1b","Type":"ContainerStarted","Data":"d962ae9702a71287902cd5daadc57616ed74af0f3a892ce1a25a3f4cb9778e2f"} Jul 07 11:41:43.270129 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.270104 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:43.272521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.272293 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" containerID="04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab" exitCode=0 Jul 07 11:41:43.272521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.272329 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7766875f79-zmv45" Jul 07 11:41:43.272521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.272338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7766875f79-zmv45" event={"ID":"0f7a8dfd-113d-4e30-9a4a-927004d0e035","Type":"ContainerDied","Data":"04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab"} Jul 07 11:41:43.272521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.272373 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7766875f79-zmv45" event={"ID":"0f7a8dfd-113d-4e30-9a4a-927004d0e035","Type":"ContainerDied","Data":"c2f8a6354d234fe4f841ff3482b6156e8745f1a11a28074ea0fd45a9008b39a4"} Jul 07 11:41:43.272521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.272399 2580 scope.go:117] "RemoveContainer" containerID="04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab" Jul 07 11:41:43.276531 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.276506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-tdg97" Jul 07 11:41:43.286698 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.286678 2580 scope.go:117] "RemoveContainer" containerID="04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab" Jul 07 11:41:43.287035 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:41:43.287012 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab\": container with ID starting with 04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab not found: ID does not exist" containerID="04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab" Jul 07 11:41:43.287120 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.287049 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab"} err="failed to get container status \"04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab\": rpc error: code = NotFound desc = could not find container \"04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab\": container with ID starting with 04ad2a219abc051b85b34437463728406588cd3d41a058f21b74d9407741d5ab not found: ID does not exist" Jul 07 11:41:43.291741 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.291621 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-tdg97" podStartSLOduration=1.247267272 podStartE2EDuration="19.291607623s" totalImagesPullingTime="18.044340351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:43.289689476 +0000 UTC m=+92.133413577" watchObservedRunningTime="2026-07-07 11:41:43.291607623 +0000 UTC m=+92.135331723" Jul 07 11:41:43.309496 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.309471 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7766875f79-zmv45"] Jul 07 11:41:43.312466 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.312443 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7766875f79-zmv45"] Jul 07 11:41:43.837786 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:43.837754 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" path="/var/lib/kubelet/pods/0f7a8dfd-113d-4e30-9a4a-927004d0e035/volumes" Jul 07 11:41:47.300483 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:47.300448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} Jul 07 11:41:47.300947 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:47.300527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} Jul 07 11:41:49.313364 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:49.313332 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} Jul 07 11:41:49.313765 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:49.313400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} Jul 07 11:41:49.313765 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:49.313439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} Jul 07 11:41:50.322061 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:50.322025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerStarted","Data":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} Jul 07 11:41:50.357795 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:50.357738 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.063220038 podStartE2EDuration="18.357717769s" totalImagesPullingTime="14.271384303s" totalInitContainerRuntime="23.113428ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:50.356392899 +0000 UTC m=+99.200117038" watchObservedRunningTime="2026-07-07 11:41:50.357717769 +0000 UTC m=+99.201441870" Jul 07 11:41:53.044750 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:53.044709 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:41:54.164638 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:41:54.164609 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qdr6z" Jul 07 11:42:04.481150 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:04.481123 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wcrp5_4aac21db-70cd-4e10-b7ff-700f56b241ba/serve-healthcheck-canary/0.log" Jul 07 11:42:33.045409 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:33.045368 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:33.064069 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:33.064036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:33.488709 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:33.488685 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:51.149278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.149239 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:42:51.149824 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.149628 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="prometheus" containerID="cri-o://80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" gracePeriod=600 Jul 07 11:42:51.150017 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.149990 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" gracePeriod=600 Jul 07 11:42:51.150129 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.150106 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy" containerID="cri-o://c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" gracePeriod=600 Jul 07 11:42:51.151591 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.150338 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-web" containerID="cri-o://db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" gracePeriod=600 Jul 07 11:42:51.152283 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.150382 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="thanos-sidecar" containerID="cri-o://4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" gracePeriod=600 Jul 07 11:42:51.152283 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.150405 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="config-reloader" containerID="cri-o://bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" gracePeriod=600 Jul 07 11:42:51.390567 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.390541 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548000 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" exitCode=0 Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548025 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" exitCode=0 Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548032 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" exitCode=0 Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548038 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" exitCode=0 Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548044 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" exitCode=0 Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548049 2580 generic.go:356] "Generic (PLEG): container finished" podID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" exitCode=0 Jul 07 11:42:51.548088 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548085 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548090 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548223 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548323 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548382 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} Jul 07 11:42:51.548562 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.548419 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ac05307f-6e1c-43ea-a139-61f19d0374f6","Type":"ContainerDied","Data":"d925c40d255e177b6a0b29c0db660e786585b888fb8f5f881b5619d4f2bdfdc2"} Jul 07 11:42:51.555513 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.555496 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.563101 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.563086 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.569750 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.569735 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.576695 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.576679 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.581788 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.581767 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-grpc-tls\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.581895 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.581804 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfd2s\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-kube-api-access-zfd2s\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.581895 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.581829 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-tls\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.581895 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.581849 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-trusted-ca-bundle\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.581895 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.581876 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-tls-assets\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582275 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582249 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-0\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582383 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582288 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.582383 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582301 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-kubelet-serving-ca-bundle\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582383 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582365 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-serving-certs-ca-bundle\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582553 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582405 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582553 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582430 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-thanos-prometheus-http-client-file\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582652 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582554 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-web-config\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582652 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582603 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-kube-rbac-proxy\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582652 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582622 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.582652 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582642 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-2\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582855 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582670 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582855 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582706 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-db\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582855 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582740 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-metrics-client-certs\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582855 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582771 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-config-out\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582855 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582803 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-metrics-client-ca\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.582855 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582831 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-config\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.583149 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.582860 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-1\") pod \"ac05307f-6e1c-43ea-a139-61f19d0374f6\" (UID: \"ac05307f-6e1c-43ea-a139-61f19d0374f6\") " Jul 07 11:42:51.583149 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.583143 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.583277 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.583165 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.583720 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.583403 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.583720 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.583704 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.584092 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.583902 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.584092 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.584039 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.584912 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.584890 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.585119 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.585100 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.585420 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.585401 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-kube-api-access-zfd2s" (OuterVolumeSpecName: "kube-api-access-zfd2s") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "kube-api-access-zfd2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:42:51.585726 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.585696 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:51.586779 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.586750 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.586870 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.586801 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:51.587235 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.587207 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.587347 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.587261 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.587347 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.587291 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:42:51.587534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.587503 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.587643 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.587538 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-config-out" (OuterVolumeSpecName: "config-out") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:51.588055 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.588034 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-config" (OuterVolumeSpecName: "config") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.588675 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.588637 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.588777 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.588689 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.592974 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.592951 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.596590 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.596570 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-web-config" (OuterVolumeSpecName: "web-config") pod "ac05307f-6e1c-43ea-a139-61f19d0374f6" (UID: "ac05307f-6e1c-43ea-a139-61f19d0374f6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:42:51.599317 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.599302 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.599558 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.599540 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.599611 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.599565 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} err="failed to get container status \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" Jul 07 11:42:51.599611 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.599585 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.599823 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.599805 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.599869 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.599828 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} err="failed to get container status \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" Jul 07 11:42:51.599869 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.599842 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.600044 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.600029 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.600085 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600047 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} err="failed to get container status \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" Jul 07 11:42:51.600085 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600061 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.600320 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.600303 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.600395 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600323 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} err="failed to get container status \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" Jul 07 11:42:51.600395 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600337 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.600581 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.600553 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.600623 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600583 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} err="failed to get container status \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" Jul 07 11:42:51.600623 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600595 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.600812 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.600797 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.600848 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600818 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} err="failed to get container status \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" Jul 07 11:42:51.600848 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.600832 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.601039 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:42:51.601023 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.601083 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601043 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} err="failed to get container status \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" Jul 07 11:42:51.601083 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601056 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.601283 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601266 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} err="failed to get container status \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" Jul 07 11:42:51.601332 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601284 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.601493 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601478 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} err="failed to get container status \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" Jul 07 11:42:51.601540 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601494 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.601730 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601711 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} err="failed to get container status \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" Jul 07 11:42:51.601791 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601733 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.601964 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601948 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} err="failed to get container status \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" Jul 07 11:42:51.602005 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.601964 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.602169 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602153 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} err="failed to get container status \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" Jul 07 11:42:51.602240 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602169 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.602429 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602408 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} err="failed to get container status \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" Jul 07 11:42:51.602474 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602430 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.602640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602626 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} err="failed to get container status \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" Jul 07 11:42:51.602693 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602641 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.602841 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602827 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} err="failed to get container status \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" Jul 07 11:42:51.602886 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602842 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.603000 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.602983 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} err="failed to get container status \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" Jul 07 11:42:51.603041 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603002 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.603222 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603205 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} err="failed to get container status \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" Jul 07 11:42:51.603222 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603222 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.603416 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603398 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} err="failed to get container status \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" Jul 07 11:42:51.603481 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603417 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.603640 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603625 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} err="failed to get container status \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" Jul 07 11:42:51.603692 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603641 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.603838 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603823 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} err="failed to get container status \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" Jul 07 11:42:51.603885 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.603839 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.604040 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604024 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} err="failed to get container status \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" Jul 07 11:42:51.604084 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604040 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.604243 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604228 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} err="failed to get container status \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" Jul 07 11:42:51.604243 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604243 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.604444 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604427 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} err="failed to get container status \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" Jul 07 11:42:51.604507 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604446 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.604676 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604660 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} err="failed to get container status \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" Jul 07 11:42:51.604720 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604676 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.604881 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604859 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} err="failed to get container status \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" Jul 07 11:42:51.604947 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.604883 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.605092 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605074 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} err="failed to get container status \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" Jul 07 11:42:51.605136 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605094 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.605305 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605289 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} err="failed to get container status \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" Jul 07 11:42:51.605354 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605305 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.605506 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605488 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} err="failed to get container status \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" Jul 07 11:42:51.605574 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605507 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.605709 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605692 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} err="failed to get container status \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" Jul 07 11:42:51.605757 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605709 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.605898 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605882 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} err="failed to get container status \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" Jul 07 11:42:51.605938 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.605898 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.606110 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606094 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} err="failed to get container status \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" Jul 07 11:42:51.606175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606110 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.606351 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606334 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} err="failed to get container status \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" Jul 07 11:42:51.606424 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606350 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.606580 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606565 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} err="failed to get container status \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" Jul 07 11:42:51.606580 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606579 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.606764 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606748 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} err="failed to get container status \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" Jul 07 11:42:51.606809 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606766 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.606950 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606936 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} err="failed to get container status \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" Jul 07 11:42:51.606994 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.606951 2580 scope.go:117] "RemoveContainer" containerID="5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106" Jul 07 11:42:51.607147 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607133 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106"} err="failed to get container status \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": rpc error: code = NotFound desc = could not find container \"5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106\": container with ID starting with 5a2e2407fd016cfd175680379cf260374dd4f566ca1d91ab6062440f67f9a106 not found: ID does not exist" Jul 07 11:42:51.607215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607147 2580 scope.go:117] "RemoveContainer" containerID="c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185" Jul 07 11:42:51.607327 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607312 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185"} err="failed to get container status \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": rpc error: code = NotFound desc = could not find container \"c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185\": container with ID starting with c2c46c167fceee6fbda762a56da7b3132319e2389bed87f5909688fc416bb185 not found: ID does not exist" Jul 07 11:42:51.607365 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607328 2580 scope.go:117] "RemoveContainer" containerID="db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2" Jul 07 11:42:51.607471 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607451 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2"} err="failed to get container status \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": rpc error: code = NotFound desc = could not find container \"db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2\": container with ID starting with db1cbc9f8f8ea6c4db86b7a40e1fed79a331be04c5a5fa15588cb66620f74ab2 not found: ID does not exist" Jul 07 11:42:51.607471 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607468 2580 scope.go:117] "RemoveContainer" containerID="4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a" Jul 07 11:42:51.607630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607613 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a"} err="failed to get container status \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": rpc error: code = NotFound desc = could not find container \"4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a\": container with ID starting with 4087c5c64c3500df30b2c366e1eda35715ee7bf5e41e2bdc4d6b1c134b5d262a not found: ID does not exist" Jul 07 11:42:51.607677 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607632 2580 scope.go:117] "RemoveContainer" containerID="bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48" Jul 07 11:42:51.607802 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607786 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48"} err="failed to get container status \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": rpc error: code = NotFound desc = could not find container \"bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48\": container with ID starting with bc29d905e5cd3ab29029813e0499caaac74290ec83a5c2bad3c4c3f507a04d48 not found: ID does not exist" Jul 07 11:42:51.607844 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607801 2580 scope.go:117] "RemoveContainer" containerID="80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13" Jul 07 11:42:51.608000 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.607984 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13"} err="failed to get container status \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": rpc error: code = NotFound desc = could not find container \"80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13\": container with ID starting with 80b13b64cef5197a98afa5ccee40537a62a0c9c175870632d6b8d94473222c13 not found: ID does not exist" Jul 07 11:42:51.608048 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.608000 2580 scope.go:117] "RemoveContainer" containerID="f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8" Jul 07 11:42:51.608216 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.608182 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8"} err="failed to get container status \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": rpc error: code = NotFound desc = could not find container \"f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8\": container with ID starting with f1b7cb7607684b769b9bcaf0271671a453936b5aa133ee678e4e700d432a78b8 not found: ID does not exist" Jul 07 11:42:51.683675 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683636 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-metrics-client-ca\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683675 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683667 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-config\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683675 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683680 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683696 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-grpc-tls\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683709 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfd2s\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-kube-api-access-zfd2s\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683719 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683729 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ac05307f-6e1c-43ea-a139-61f19d0374f6-tls-assets\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683736 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683745 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683756 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683770 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683784 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-web-config\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683792 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-kube-rbac-proxy\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683799 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683808 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683817 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-prometheus-k8s-db\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683825 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ac05307f-6e1c-43ea-a139-61f19d0374f6-secret-metrics-client-certs\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.683893 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.683833 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ac05307f-6e1c-43ea-a139-61f19d0374f6-config-out\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:42:51.869257 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.869226 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:42:51.873714 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.873685 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:42:51.905647 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905620 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:42:51.905866 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905854 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" containerName="registry" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905867 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" containerName="registry" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905879 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="prometheus" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905885 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="prometheus" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905891 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905896 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905904 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="init-config-reloader" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905910 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="init-config-reloader" Jul 07 11:42:51.905915 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905918 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-thanos" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905924 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-thanos" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905931 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-web" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905936 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-web" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905947 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="config-reloader" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905952 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="config-reloader" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905959 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="thanos-sidecar" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.905966 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="thanos-sidecar" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906003 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-web" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906009 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f7a8dfd-113d-4e30-9a4a-927004d0e035" containerName="registry" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906015 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906020 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="kube-rbac-proxy-thanos" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906025 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="prometheus" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906030 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="thanos-sidecar" Jul 07 11:42:51.906175 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.906035 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" containerName="config-reloader" Jul 07 11:42:51.909569 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.909553 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:51.912046 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912024 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:42:51.912046 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912039 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:42:51.912259 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912059 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:42:51.912259 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912040 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sbv7b\"" Jul 07 11:42:51.912259 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:42:51.912259 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912038 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:42:51.912465 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912337 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:42:51.912624 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912537 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:42:51.912624 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912558 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:42:51.912624 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912585 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:42:51.912803 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912647 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:42:51.912803 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:42:51.912909 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912856 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-29jv15socv5rf\"" Jul 07 11:42:51.912909 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.912886 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:42:51.913056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.913037 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:42:51.917568 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.916283 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:42:51.921210 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.921172 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:42:51.926727 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:51.926708 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:42:52.086707 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-config-out\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.086880 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.086880 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.086880 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.086880 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.086982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-web-config\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg8p2\" (UniqueName: \"kubernetes.io/projected/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-kube-api-access-sg8p2\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087322 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087533 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.087533 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.087391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-config\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188003 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.187974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-config\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-config-out\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-web-config\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.188403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.189178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg8p2\" (UniqueName: \"kubernetes.io/projected/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-kube-api-access-sg8p2\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.189178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.188449 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.189178 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.189011 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.189353 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.189281 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.189353 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.189310 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.189833 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.189805 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.191134 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.191104 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.191277 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.191145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.191346 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.191305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-config-out\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.191664 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.191639 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.191811 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.191775 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.191811 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.191801 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.192215 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.192150 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.192300 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.192279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.192787 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.192646 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.192919 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.192900 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.193149 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.193126 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.193527 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.193509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-config\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.193822 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.193807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.194246 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.194231 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-web-config\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.195453 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.195433 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.200256 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.200233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg8p2\" (UniqueName: \"kubernetes.io/projected/22f3b1cd-2b3d-4800-bb9b-5ee2337a880a-kube-api-access-sg8p2\") pod \"prometheus-k8s-0\" (UID: \"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.222118 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.222097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:42:52.358283 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.358259 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:42:52.360588 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:42:52.360566 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f3b1cd_2b3d_4800_bb9b_5ee2337a880a.slice/crio-32c426645145608909d329f73e40cf638afe58b5c79e016895d968214dabf1a2 WatchSource:0}: Error finding container 32c426645145608909d329f73e40cf638afe58b5c79e016895d968214dabf1a2: Status 404 returned error can't find the container with id 32c426645145608909d329f73e40cf638afe58b5c79e016895d968214dabf1a2 Jul 07 11:42:52.553066 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.553038 2580 generic.go:356] "Generic (PLEG): container finished" podID="22f3b1cd-2b3d-4800-bb9b-5ee2337a880a" containerID="c448929e20ea4a6971900aefe45fd21eb2229b4fda3a74af824c3c0f53dfc428" exitCode=0 Jul 07 11:42:52.553256 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.553127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerDied","Data":"c448929e20ea4a6971900aefe45fd21eb2229b4fda3a74af824c3c0f53dfc428"} Jul 07 11:42:52.553256 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:52.553238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"32c426645145608909d329f73e40cf638afe58b5c79e016895d968214dabf1a2"} Jul 07 11:42:53.565320 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.565287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"8a9f30aeadc8c41e05f40e901287bdee63c742cb05a3e6f9d24289697da77669"} Jul 07 11:42:53.565701 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.565369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"19adc6366f5dd379386441f47e7900e229dc10319c75289a79f38fc158011bff"} Jul 07 11:42:53.565701 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.565395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"82aaa1dedd3589ef2dcbe2a7b9c5e5cad6749dc23cd186ca7603f075a9d282d4"} Jul 07 11:42:53.565701 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.565421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"f43e7fc0a9ca529dbbd15f41dd14261a19e939812f3ec458432829eff67b798e"} Jul 07 11:42:53.565701 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.565443 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"7ba36fc6be59e6a19eb1a428f10c93127ae58badd7c42c466751623eb3b78c84"} Jul 07 11:42:53.565701 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.565465 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"22f3b1cd-2b3d-4800-bb9b-5ee2337a880a","Type":"ContainerStarted","Data":"fb681feda9cc5b9c1ef754c61079352ee23340a1db6930fc816e83b0f6d4b1ab"} Jul 07 11:42:53.834760 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:53.834683 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac05307f-6e1c-43ea-a139-61f19d0374f6" path="/var/lib/kubelet/pods/ac05307f-6e1c-43ea-a139-61f19d0374f6/volumes" Jul 07 11:42:57.222844 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:42:57.222812 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:43:52.222983 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:43:52.222954 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:43:52.237914 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:43:52.237893 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:43:52.268140 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:43:52.268088 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=61.251318971 podStartE2EDuration="1m1.268075271s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.7563ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:53.606130886 +0000 UTC m=+162.449855002" watchObservedRunningTime="2026-07-07 11:43:52.268075271 +0000 UTC m=+221.111799370" Jul 07 11:43:52.795920 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:43:52.795888 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:45:11.620416 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:45:11.620391 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 11:48:27.467994 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.467959 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-dqnnw"] Jul 07 11:48:27.470103 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.470086 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.472090 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.472064 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 07 11:48:27.472223 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.472161 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 07 11:48:27.472280 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.472254 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-89472\"" Jul 07 11:48:27.472618 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.472597 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 07 11:48:27.480472 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.480449 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-dqnnw"] Jul 07 11:48:27.562267 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.562233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-model-cache\") pod \"s3-init-dqnnw\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.562389 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.562334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmbvh\" (UniqueName: \"kubernetes.io/projected/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-kube-api-access-rmbvh\") pod \"s3-init-dqnnw\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.663663 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.663635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmbvh\" (UniqueName: \"kubernetes.io/projected/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-kube-api-access-rmbvh\") pod \"s3-init-dqnnw\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.663751 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.663676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-model-cache\") pod \"s3-init-dqnnw\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.668472 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.664455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-model-cache\") pod \"s3-init-dqnnw\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.672056 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.672032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmbvh\" (UniqueName: \"kubernetes.io/projected/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-kube-api-access-rmbvh\") pod \"s3-init-dqnnw\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.778600 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.778531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dqnnw" Jul 07 11:48:27.899672 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.899641 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-dqnnw"] Jul 07 11:48:27.903343 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:48:27.903318 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ab92d5f_30d1_4e02_99e5_b34210e2ce1d.slice/crio-e6516df294ecb32b576cc2192b79c96694b244fa5346d8d2656950c5b86b875f WatchSource:0}: Error finding container e6516df294ecb32b576cc2192b79c96694b244fa5346d8d2656950c5b86b875f: Status 404 returned error can't find the container with id e6516df294ecb32b576cc2192b79c96694b244fa5346d8d2656950c5b86b875f Jul 07 11:48:27.905156 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:27.905139 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:48:28.732539 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:28.732501 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dqnnw" event={"ID":"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d","Type":"ContainerStarted","Data":"e6516df294ecb32b576cc2192b79c96694b244fa5346d8d2656950c5b86b875f"} Jul 07 11:48:31.747826 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:31.747790 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dqnnw" event={"ID":"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d","Type":"ContainerStarted","Data":"c60547a994cc6957592becefd5bfb08344b2762c1da816c1c5400db21a2cb747"} Jul 07 11:48:33.757077 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:33.757043 2580 generic.go:356] "Generic (PLEG): container finished" podID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerID="c60547a994cc6957592becefd5bfb08344b2762c1da816c1c5400db21a2cb747" exitCode=0 Jul 07 11:48:33.757487 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:33.757104 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dqnnw" event={"ID":"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d","Type":"ContainerDied","Data":"c60547a994cc6957592becefd5bfb08344b2762c1da816c1c5400db21a2cb747"} Jul 07 11:48:38.780118 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:38.780083 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dqnnw" event={"ID":"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d","Type":"ContainerStarted","Data":"a9ca0b90c102a0e40998147716b866392cb9f2f1e232c7d19ffb18b7b353ee41"} Jul 07 11:48:38.796851 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:48:38.796809 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-dqnnw" podStartSLOduration=1.954119747 podStartE2EDuration="11.796793882s" totalImagesPullingTime="8.218294004s" totalInitContainerRuntime="1.624380131s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:38.795361927 +0000 UTC m=+507.639086040" watchObservedRunningTime="2026-07-07 11:48:38.796793882 +0000 UTC m=+507.640517981" Jul 07 11:49:44.011534 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:44.011500 2580 generic.go:356] "Generic (PLEG): container finished" podID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerID="a9ca0b90c102a0e40998147716b866392cb9f2f1e232c7d19ffb18b7b353ee41" exitCode=0 Jul 07 11:49:44.011935 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:44.011573 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dqnnw" event={"ID":"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d","Type":"ContainerDied","Data":"a9ca0b90c102a0e40998147716b866392cb9f2f1e232c7d19ffb18b7b353ee41"} Jul 07 11:49:45.135069 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.135048 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dqnnw" Jul 07 11:49:45.172949 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.172918 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmbvh\" (UniqueName: \"kubernetes.io/projected/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-kube-api-access-rmbvh\") pod \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " Jul 07 11:49:45.172949 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.172951 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-model-cache\") pod \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\" (UID: \"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d\") " Jul 07 11:49:45.174948 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.174919 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-kube-api-access-rmbvh" (OuterVolumeSpecName: "kube-api-access-rmbvh") pod "8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" (UID: "8ab92d5f-30d1-4e02-99e5-b34210e2ce1d"). InnerVolumeSpecName "kube-api-access-rmbvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:49:45.191403 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.191380 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" (UID: "8ab92d5f-30d1-4e02-99e5-b34210e2ce1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:49:45.273637 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.273585 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmbvh\" (UniqueName: \"kubernetes.io/projected/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-kube-api-access-rmbvh\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:49:45.273637 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:45.273604 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab92d5f-30d1-4e02-99e5-b34210e2ce1d-model-cache\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:49:46.020207 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:46.020157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dqnnw" event={"ID":"8ab92d5f-30d1-4e02-99e5-b34210e2ce1d","Type":"ContainerDied","Data":"e6516df294ecb32b576cc2192b79c96694b244fa5346d8d2656950c5b86b875f"} Jul 07 11:49:46.020371 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:46.020215 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6516df294ecb32b576cc2192b79c96694b244fa5346d8d2656950c5b86b875f" Jul 07 11:49:46.020371 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:49:46.020242 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dqnnw" Jul 07 11:51:53.124950 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.124923 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn"] Jul 07 11:51:53.125357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.125157 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerName="download-hf-model" Jul 07 11:51:53.125357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.125167 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerName="download-hf-model" Jul 07 11:51:53.125357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.125177 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerName="s3-init" Jul 07 11:51:53.125357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.125182 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerName="s3-init" Jul 07 11:51:53.125357 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.125255 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab92d5f-30d1-4e02-99e5-b34210e2ce1d" containerName="s3-init" Jul 07 11:51:53.128089 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.128070 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.131002 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.130958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-lhlhn\"" Jul 07 11:51:53.131002 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.130971 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 07 11:51:53.131002 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.131000 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 07 11:51:53.131293 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.130971 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-fea16-predictor-serving-cert\"" Jul 07 11:51:53.131293 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.131075 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 07 11:51:53.131293 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.131151 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-fea16-kube-rbac-proxy-sar-config\"" Jul 07 11:51:53.131293 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.131258 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 07 11:51:53.140565 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.140545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn"] Jul 07 11:51:53.230987 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.230968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkrll\" (UniqueName: \"kubernetes.io/projected/333efe98-d5df-44da-adbe-ad3c413e1780-kube-api-access-nkrll\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.231089 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.231000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/333efe98-d5df-44da-adbe-ad3c413e1780-proxy-tls\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.231089 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.231020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/333efe98-d5df-44da-adbe-ad3c413e1780-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.231089 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.231082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.231190 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.231137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-cabundle-cert\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.331974 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.331954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-cabundle-cert\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332065 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.331999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkrll\" (UniqueName: \"kubernetes.io/projected/333efe98-d5df-44da-adbe-ad3c413e1780-kube-api-access-nkrll\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332065 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.332031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/333efe98-d5df-44da-adbe-ad3c413e1780-proxy-tls\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332065 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.332056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/333efe98-d5df-44da-adbe-ad3c413e1780-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332244 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.332099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332469 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.332452 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/333efe98-d5df-44da-adbe-ad3c413e1780-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332767 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.332748 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-cabundle-cert\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.332813 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.332773 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.334499 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.334477 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/333efe98-d5df-44da-adbe-ad3c413e1780-proxy-tls\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.340727 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.340709 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkrll\" (UniqueName: \"kubernetes.io/projected/333efe98-d5df-44da-adbe-ad3c413e1780-kube-api-access-nkrll\") pod \"isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.437965 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.437914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:51:53.567129 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:53.567106 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn"] Jul 07 11:51:53.569596 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:51:53.569570 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333efe98_d5df_44da_adbe_ad3c413e1780.slice/crio-a4b18adad5a79f14a03971b6f0e500303ce0f41fde255ec6b0c3d7c65e8fd252 WatchSource:0}: Error finding container a4b18adad5a79f14a03971b6f0e500303ce0f41fde255ec6b0c3d7c65e8fd252: Status 404 returned error can't find the container with id a4b18adad5a79f14a03971b6f0e500303ce0f41fde255ec6b0c3d7c65e8fd252 Jul 07 11:51:54.476460 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:54.476426 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerStarted","Data":"e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50"} Jul 07 11:51:54.476829 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:54.476485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerStarted","Data":"a4b18adad5a79f14a03971b6f0e500303ce0f41fde255ec6b0c3d7c65e8fd252"} Jul 07 11:51:57.488650 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:57.488623 2580 generic.go:356] "Generic (PLEG): container finished" podID="333efe98-d5df-44da-adbe-ad3c413e1780" containerID="e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50" exitCode=0 Jul 07 11:51:57.488932 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:51:57.488690 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerDied","Data":"e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50"} Jul 07 11:52:15.565014 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:15.564976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerStarted","Data":"9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f"} Jul 07 11:52:18.577445 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:18.577405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerStarted","Data":"117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f"} Jul 07 11:52:18.577919 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:18.577506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:52:18.577919 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:18.577658 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:52:18.578865 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:18.578842 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:52:18.597983 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:18.597942 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podStartSLOduration=1.229486159 podStartE2EDuration="25.597928949s" totalImagesPullingTime="20.521364929s" totalInitContainerRuntime="3.847077861s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:51:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:52:18.596805823 +0000 UTC m=+727.440529932" watchObservedRunningTime="2026-07-07 11:52:18.597928949 +0000 UTC m=+727.441653045" Jul 07 11:52:19.581129 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:19.581090 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:52:24.586274 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:24.586246 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:52:24.586849 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:24.586732 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:52:34.587278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:34.587241 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:52:44.586687 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:44.586642 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:52:54.587065 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:52:54.587026 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:53:04.587060 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:04.587016 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:53:14.587362 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:14.587331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:53:43.424712 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.424632 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz"] Jul 07 11:53:43.426813 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.426798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.429230 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.429208 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-e5dd1-kube-rbac-proxy-sar-config\"" Jul 07 11:53:43.429230 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.429225 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-serving-cert\"" Jul 07 11:53:43.442058 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.442032 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz"] Jul 07 11:53:43.513159 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.513131 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn"] Jul 07 11:53:43.513418 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.513392 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" containerID="cri-o://9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f" gracePeriod=30 Jul 07 11:53:43.513544 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.513404 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kube-rbac-proxy" containerID="cri-o://117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f" gracePeriod=30 Jul 07 11:53:43.613995 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.613970 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz5pd\" (UniqueName: \"kubernetes.io/projected/7bed052b-565d-42d9-86fc-d7384075af91-kube-api-access-nz5pd\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.614121 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.614019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.614121 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.614094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7bed052b-565d-42d9-86fc-d7384075af91-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.614237 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.614152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7bed052b-565d-42d9-86fc-d7384075af91-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.614237 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.614217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715148 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715148 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715124 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7bed052b-565d-42d9-86fc-d7384075af91-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715341 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7bed052b-565d-42d9-86fc-d7384075af91-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715341 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715341 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz5pd\" (UniqueName: \"kubernetes.io/projected/7bed052b-565d-42d9-86fc-d7384075af91-kube-api-access-nz5pd\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715643 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7bed052b-565d-42d9-86fc-d7384075af91-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715829 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715811 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.715870 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.715813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.717539 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.717519 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7bed052b-565d-42d9-86fc-d7384075af91-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.724746 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.724725 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz5pd\" (UniqueName: \"kubernetes.io/projected/7bed052b-565d-42d9-86fc-d7384075af91-kube-api-access-nz5pd\") pod \"isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.740759 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.740742 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:43.867631 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.867554 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz"] Jul 07 11:53:43.870052 ip-10-0-138-138 kubenswrapper[2580]: W0707 11:53:43.870028 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bed052b_565d_42d9_86fc_d7384075af91.slice/crio-f9a4f329571a214566efebf10a17a6f8594a244185a81d97b0b42502229e2617 WatchSource:0}: Error finding container f9a4f329571a214566efebf10a17a6f8594a244185a81d97b0b42502229e2617: Status 404 returned error can't find the container with id f9a4f329571a214566efebf10a17a6f8594a244185a81d97b0b42502229e2617 Jul 07 11:53:43.887654 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.887632 2580 generic.go:356] "Generic (PLEG): container finished" podID="333efe98-d5df-44da-adbe-ad3c413e1780" containerID="117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f" exitCode=2 Jul 07 11:53:43.887725 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.887700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerDied","Data":"117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f"} Jul 07 11:53:43.888763 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:43.888743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerStarted","Data":"f9a4f329571a214566efebf10a17a6f8594a244185a81d97b0b42502229e2617"} Jul 07 11:53:44.582174 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:44.582134 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused" Jul 07 11:53:44.586715 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:44.586688 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused" Jul 07 11:53:44.894122 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:44.894086 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerStarted","Data":"c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00"} Jul 07 11:53:46.851212 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.851167 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:53:46.903273 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.903208 2580 generic.go:356] "Generic (PLEG): container finished" podID="333efe98-d5df-44da-adbe-ad3c413e1780" containerID="9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f" exitCode=0 Jul 07 11:53:46.903273 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.903251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerDied","Data":"9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f"} Jul 07 11:53:46.903273 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.903260 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" Jul 07 11:53:46.903539 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.903298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn" event={"ID":"333efe98-d5df-44da-adbe-ad3c413e1780","Type":"ContainerDied","Data":"a4b18adad5a79f14a03971b6f0e500303ce0f41fde255ec6b0c3d7c65e8fd252"} Jul 07 11:53:46.903539 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.903327 2580 scope.go:117] "RemoveContainer" containerID="117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f" Jul 07 11:53:46.910496 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.910482 2580 scope.go:117] "RemoveContainer" containerID="9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f" Jul 07 11:53:46.917084 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.917065 2580 scope.go:117] "RemoveContainer" containerID="e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50" Jul 07 11:53:46.925664 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.925649 2580 scope.go:117] "RemoveContainer" containerID="117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f" Jul 07 11:53:46.925912 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:53:46.925890 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f\": container with ID starting with 117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f not found: ID does not exist" containerID="117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f" Jul 07 11:53:46.925969 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.925924 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f"} err="failed to get container status \"117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f\": rpc error: code = NotFound desc = could not find container \"117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f\": container with ID starting with 117f9b9aa898eb113c3228c44990c7c04f803ca476955a72b1fa08933057bb7f not found: ID does not exist" Jul 07 11:53:46.925969 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.925941 2580 scope.go:117] "RemoveContainer" containerID="9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f" Jul 07 11:53:46.926182 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:53:46.926164 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f\": container with ID starting with 9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f not found: ID does not exist" containerID="9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f" Jul 07 11:53:46.926278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.926207 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f"} err="failed to get container status \"9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f\": rpc error: code = NotFound desc = could not find container \"9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f\": container with ID starting with 9c58a08b8d05f1d390c9a9f645c383ef82e135a80a14d3f95e0410211191a34f not found: ID does not exist" Jul 07 11:53:46.926278 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.926231 2580 scope.go:117] "RemoveContainer" containerID="e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50" Jul 07 11:53:46.926436 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:53:46.926416 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50\": container with ID starting with e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50 not found: ID does not exist" containerID="e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50" Jul 07 11:53:46.926471 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:46.926441 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50"} err="failed to get container status \"e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50\": rpc error: code = NotFound desc = could not find container \"e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50\": container with ID starting with e05889dcdd4476f874d6488e3f0e85cd11ea62be02443b7a6b7834ca81880f50 not found: ID does not exist" Jul 07 11:53:47.045904 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.045885 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/333efe98-d5df-44da-adbe-ad3c413e1780-proxy-tls\") pod \"333efe98-d5df-44da-adbe-ad3c413e1780\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " Jul 07 11:53:47.045982 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.045921 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkrll\" (UniqueName: \"kubernetes.io/projected/333efe98-d5df-44da-adbe-ad3c413e1780-kube-api-access-nkrll\") pod \"333efe98-d5df-44da-adbe-ad3c413e1780\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " Jul 07 11:53:47.045982 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.045952 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/333efe98-d5df-44da-adbe-ad3c413e1780-kserve-provision-location\") pod \"333efe98-d5df-44da-adbe-ad3c413e1780\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " Jul 07 11:53:47.046099 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.046008 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-cabundle-cert\") pod \"333efe98-d5df-44da-adbe-ad3c413e1780\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " Jul 07 11:53:47.046099 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.046028 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-kube-rbac-proxy-sar-config\") pod \"333efe98-d5df-44da-adbe-ad3c413e1780\" (UID: \"333efe98-d5df-44da-adbe-ad3c413e1780\") " Jul 07 11:53:47.046350 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.046288 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333efe98-d5df-44da-adbe-ad3c413e1780-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "333efe98-d5df-44da-adbe-ad3c413e1780" (UID: "333efe98-d5df-44da-adbe-ad3c413e1780"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:53:47.046425 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.046347 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "333efe98-d5df-44da-adbe-ad3c413e1780" (UID: "333efe98-d5df-44da-adbe-ad3c413e1780"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:53:47.046425 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.046373 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "333efe98-d5df-44da-adbe-ad3c413e1780" (UID: "333efe98-d5df-44da-adbe-ad3c413e1780"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:53:47.048018 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.047998 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/333efe98-d5df-44da-adbe-ad3c413e1780-kube-api-access-nkrll" (OuterVolumeSpecName: "kube-api-access-nkrll") pod "333efe98-d5df-44da-adbe-ad3c413e1780" (UID: "333efe98-d5df-44da-adbe-ad3c413e1780"). InnerVolumeSpecName "kube-api-access-nkrll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:53:47.048091 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.048034 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/333efe98-d5df-44da-adbe-ad3c413e1780-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "333efe98-d5df-44da-adbe-ad3c413e1780" (UID: "333efe98-d5df-44da-adbe-ad3c413e1780"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:53:47.147393 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.147373 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-cabundle-cert\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:53:47.147393 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.147393 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/333efe98-d5df-44da-adbe-ad3c413e1780-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:53:47.147521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.147403 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/333efe98-d5df-44da-adbe-ad3c413e1780-proxy-tls\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:53:47.147521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.147412 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkrll\" (UniqueName: \"kubernetes.io/projected/333efe98-d5df-44da-adbe-ad3c413e1780-kube-api-access-nkrll\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:53:47.147521 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.147420 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/333efe98-d5df-44da-adbe-ad3c413e1780-kserve-provision-location\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:53:47.227279 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.227239 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn"] Jul 07 11:53:47.229788 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.229769 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-fea16-predictor-778c496df-m5wxn"] Jul 07 11:53:47.834308 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.834213 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" path="/var/lib/kubelet/pods/333efe98-d5df-44da-adbe-ad3c413e1780/volumes" Jul 07 11:53:47.909063 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.909032 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bed052b-565d-42d9-86fc-d7384075af91" containerID="c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00" exitCode=0 Jul 07 11:53:47.909437 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:47.909098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerDied","Data":"c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00"} Jul 07 11:53:48.915144 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:48.915111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerStarted","Data":"7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670"} Jul 07 11:53:48.915525 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:48.915171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerStarted","Data":"e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda"} Jul 07 11:53:48.915525 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:48.915382 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:48.933989 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:48.933940 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podStartSLOduration=2.27497156 podStartE2EDuration="5.933921919s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.658950359s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:53:48.933788152 +0000 UTC m=+817.777512275" watchObservedRunningTime="2026-07-07 11:53:48.933921919 +0000 UTC m=+817.777646046" Jul 07 11:53:49.918839 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:49.918811 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:49.919891 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:49.919867 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:53:50.922660 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:50.922622 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:53:55.927166 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:55.927135 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:53:55.927770 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:53:55.927745 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:54:05.928684 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:54:05.928641 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:54:15.928647 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:54:15.928605 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:54:25.928719 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:54:25.928676 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:54:35.928048 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:54:35.928002 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:54:45.928913 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:54:45.928883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:55:33.680784 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:33.680755 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz"] Jul 07 11:55:33.681341 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:33.681011 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" containerID="cri-o://e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda" gracePeriod=30 Jul 07 11:55:33.681341 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:33.681064 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kube-rbac-proxy" containerID="cri-o://7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670" gracePeriod=30 Jul 07 11:55:34.295549 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:34.295512 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bed052b-565d-42d9-86fc-d7384075af91" containerID="7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670" exitCode=2 Jul 07 11:55:34.295729 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:34.295588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerDied","Data":"7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670"} Jul 07 11:55:35.922815 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:35.922769 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused" Jul 07 11:55:35.928126 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:35.928096 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Jul 07 11:55:37.019275 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.019243 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:55:37.103221 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103131 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-kube-rbac-proxy-sar-config\") pod \"7bed052b-565d-42d9-86fc-d7384075af91\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " Jul 07 11:55:37.103221 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103180 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz5pd\" (UniqueName: \"kubernetes.io/projected/7bed052b-565d-42d9-86fc-d7384075af91-kube-api-access-nz5pd\") pod \"7bed052b-565d-42d9-86fc-d7384075af91\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " Jul 07 11:55:37.103404 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103234 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7bed052b-565d-42d9-86fc-d7384075af91-kserve-provision-location\") pod \"7bed052b-565d-42d9-86fc-d7384075af91\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " Jul 07 11:55:37.103404 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103283 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7bed052b-565d-42d9-86fc-d7384075af91-proxy-tls\") pod \"7bed052b-565d-42d9-86fc-d7384075af91\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " Jul 07 11:55:37.103404 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103309 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-cabundle-cert\") pod \"7bed052b-565d-42d9-86fc-d7384075af91\" (UID: \"7bed052b-565d-42d9-86fc-d7384075af91\") " Jul 07 11:55:37.103630 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103599 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bed052b-565d-42d9-86fc-d7384075af91-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7bed052b-565d-42d9-86fc-d7384075af91" (UID: "7bed052b-565d-42d9-86fc-d7384075af91"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:55:37.103752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103616 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7bed052b-565d-42d9-86fc-d7384075af91" (UID: "7bed052b-565d-42d9-86fc-d7384075af91"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:55:37.103752 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.103664 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7bed052b-565d-42d9-86fc-d7384075af91" (UID: "7bed052b-565d-42d9-86fc-d7384075af91"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:55:37.105301 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.105272 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bed052b-565d-42d9-86fc-d7384075af91-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7bed052b-565d-42d9-86fc-d7384075af91" (UID: "7bed052b-565d-42d9-86fc-d7384075af91"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:55:37.105443 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.105312 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bed052b-565d-42d9-86fc-d7384075af91-kube-api-access-nz5pd" (OuterVolumeSpecName: "kube-api-access-nz5pd") pod "7bed052b-565d-42d9-86fc-d7384075af91" (UID: "7bed052b-565d-42d9-86fc-d7384075af91"). InnerVolumeSpecName "kube-api-access-nz5pd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:55:37.204412 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.204380 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7bed052b-565d-42d9-86fc-d7384075af91-proxy-tls\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:55:37.204412 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.204405 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-cabundle-cert\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:55:37.204412 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.204414 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7bed052b-565d-42d9-86fc-d7384075af91-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:55:37.204629 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.204423 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz5pd\" (UniqueName: \"kubernetes.io/projected/7bed052b-565d-42d9-86fc-d7384075af91-kube-api-access-nz5pd\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:55:37.204629 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.204432 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7bed052b-565d-42d9-86fc-d7384075af91-kserve-provision-location\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 11:55:37.309740 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.309700 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bed052b-565d-42d9-86fc-d7384075af91" containerID="e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda" exitCode=0 Jul 07 11:55:37.309896 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.309760 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" Jul 07 11:55:37.309896 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.309762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerDied","Data":"e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda"} Jul 07 11:55:37.309896 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.309824 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz" event={"ID":"7bed052b-565d-42d9-86fc-d7384075af91","Type":"ContainerDied","Data":"f9a4f329571a214566efebf10a17a6f8594a244185a81d97b0b42502229e2617"} Jul 07 11:55:37.309896 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.309861 2580 scope.go:117] "RemoveContainer" containerID="7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670" Jul 07 11:55:37.323414 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.323395 2580 scope.go:117] "RemoveContainer" containerID="e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda" Jul 07 11:55:37.330185 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.330169 2580 scope.go:117] "RemoveContainer" containerID="c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00" Jul 07 11:55:37.335920 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.335891 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz"] Jul 07 11:55:37.337761 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.337738 2580 scope.go:117] "RemoveContainer" containerID="7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670" Jul 07 11:55:37.338021 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:55:37.338004 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670\": container with ID starting with 7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670 not found: ID does not exist" containerID="7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670" Jul 07 11:55:37.338094 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.338031 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670"} err="failed to get container status \"7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670\": rpc error: code = NotFound desc = could not find container \"7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670\": container with ID starting with 7d27204cb9e96eba784cdad2583c35f62aa25629961a260bf63d84fd16d86670 not found: ID does not exist" Jul 07 11:55:37.338094 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.338054 2580 scope.go:117] "RemoveContainer" containerID="e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda" Jul 07 11:55:37.338362 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:55:37.338325 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda\": container with ID starting with e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda not found: ID does not exist" containerID="e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda" Jul 07 11:55:37.338417 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.338374 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda"} err="failed to get container status \"e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda\": rpc error: code = NotFound desc = could not find container \"e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda\": container with ID starting with e4aa0a27a2b4b220699255d2fe90f027603ab1c78e3b6971f2f3f2122a554cda not found: ID does not exist" Jul 07 11:55:37.338417 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.338398 2580 scope.go:117] "RemoveContainer" containerID="c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00" Jul 07 11:55:37.338679 ip-10-0-138-138 kubenswrapper[2580]: E0707 11:55:37.338659 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00\": container with ID starting with c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00 not found: ID does not exist" containerID="c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00" Jul 07 11:55:37.338803 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.338687 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00"} err="failed to get container status \"c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00\": rpc error: code = NotFound desc = could not find container \"c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00\": container with ID starting with c5add07e23f8fffc2fb1d021371164ac01b26205db11536a5a2e8cca64d3ec00 not found: ID does not exist" Jul 07 11:55:37.340466 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.340443 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-e5dd1-predictor-6975b9845-2wksz"] Jul 07 11:55:37.834315 ip-10-0-138-138 kubenswrapper[2580]: I0707 11:55:37.834284 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bed052b-565d-42d9-86fc-d7384075af91" path="/var/lib/kubelet/pods/7bed052b-565d-42d9-86fc-d7384075af91/volumes" Jul 07 12:05:16.723610 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723576 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wrx5/must-gather-zprmd"] Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723858 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kube-rbac-proxy" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723872 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kube-rbac-proxy" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723885 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="storage-initializer" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723892 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="storage-initializer" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723899 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="storage-initializer" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723905 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="storage-initializer" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723917 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723923 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723930 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723935 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723946 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kube-rbac-proxy" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723953 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kube-rbac-proxy" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.723994 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kube-rbac-proxy" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.724003 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="333efe98-d5df-44da-adbe-ad3c413e1780" containerName="kserve-container" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.724008 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kube-rbac-proxy" Jul 07 12:05:16.724127 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.724016 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bed052b-565d-42d9-86fc-d7384075af91" containerName="kserve-container" Jul 07 12:05:16.726961 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.726945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:16.729165 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.729140 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wrx5\"/\"openshift-service-ca.crt\"" Jul 07 12:05:16.729618 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.729601 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wrx5\"/\"kube-root-ca.crt\"" Jul 07 12:05:16.729699 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.729608 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4wrx5\"/\"default-dockercfg-s4kdx\"" Jul 07 12:05:16.734637 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.734616 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wrx5/must-gather-zprmd"] Jul 07 12:05:16.810097 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.810069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfmn9\" (UniqueName: \"kubernetes.io/projected/ad93885a-3abd-45a0-9f14-fed7259cef28-kube-api-access-mfmn9\") pod \"must-gather-zprmd\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:16.810247 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.810144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ad93885a-3abd-45a0-9f14-fed7259cef28-must-gather-output\") pod \"must-gather-zprmd\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:16.911359 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.911319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfmn9\" (UniqueName: \"kubernetes.io/projected/ad93885a-3abd-45a0-9f14-fed7259cef28-kube-api-access-mfmn9\") pod \"must-gather-zprmd\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:16.911552 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.911416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ad93885a-3abd-45a0-9f14-fed7259cef28-must-gather-output\") pod \"must-gather-zprmd\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:16.911752 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.911730 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ad93885a-3abd-45a0-9f14-fed7259cef28-must-gather-output\") pod \"must-gather-zprmd\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:16.918696 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:16.918669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfmn9\" (UniqueName: \"kubernetes.io/projected/ad93885a-3abd-45a0-9f14-fed7259cef28-kube-api-access-mfmn9\") pod \"must-gather-zprmd\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:17.036420 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:17.036360 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:17.159946 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:17.159922 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wrx5/must-gather-zprmd"] Jul 07 12:05:17.162288 ip-10-0-138-138 kubenswrapper[2580]: W0707 12:05:17.162258 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad93885a_3abd_45a0_9f14_fed7259cef28.slice/crio-62627ec41cef8610497c09ecbd7477b765d2a7aadc68d855127f7f409b2a2c40 WatchSource:0}: Error finding container 62627ec41cef8610497c09ecbd7477b765d2a7aadc68d855127f7f409b2a2c40: Status 404 returned error can't find the container with id 62627ec41cef8610497c09ecbd7477b765d2a7aadc68d855127f7f409b2a2c40 Jul 07 12:05:17.164027 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:17.164013 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:05:17.359808 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:17.359724 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wrx5/must-gather-zprmd" event={"ID":"ad93885a-3abd-45a0-9f14-fed7259cef28","Type":"ContainerStarted","Data":"62627ec41cef8610497c09ecbd7477b765d2a7aadc68d855127f7f409b2a2c40"} Jul 07 12:05:23.387803 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:23.387763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wrx5/must-gather-zprmd" event={"ID":"ad93885a-3abd-45a0-9f14-fed7259cef28","Type":"ContainerStarted","Data":"11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09"} Jul 07 12:05:24.393809 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:24.393772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wrx5/must-gather-zprmd" event={"ID":"ad93885a-3abd-45a0-9f14-fed7259cef28","Type":"ContainerStarted","Data":"5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79"} Jul 07 12:05:24.410050 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:24.410005 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wrx5/must-gather-zprmd" podStartSLOduration=2.336155167 podStartE2EDuration="8.409990866s" totalImagesPullingTime="6.073835699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:05:24.408692807 +0000 UTC m=+1513.252416938" watchObservedRunningTime="2026-07-07 12:05:24.409990866 +0000 UTC m=+1513.253714995" Jul 07 12:05:41.466224 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:41.466175 2580 generic.go:356] "Generic (PLEG): container finished" podID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerID="11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09" exitCode=0 Jul 07 12:05:41.466574 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:41.466261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wrx5/must-gather-zprmd" event={"ID":"ad93885a-3abd-45a0-9f14-fed7259cef28","Type":"ContainerDied","Data":"11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09"} Jul 07 12:05:41.466614 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:41.466591 2580 scope.go:117] "RemoveContainer" containerID="11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09" Jul 07 12:05:42.379577 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:42.379546 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4wrx5_must-gather-zprmd_ad93885a-3abd-45a0-9f14-fed7259cef28/gather/0.log" Jul 07 12:05:45.816454 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:45.816410 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-v7v7r_1a31e3c1-5122-433a-88e1-da067af93778/global-pull-secret-syncer/0.log" Jul 07 12:05:45.937067 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:45.937031 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mhtgj_c0fb452a-c51e-4c8b-9527-c23456b6e4cb/konnectivity-agent/0.log" Jul 07 12:05:46.015286 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:46.015257 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-138.ec2.internal_ea773fa8591c47f0caeb4b4d0061f7eb/haproxy/0.log" Jul 07 12:05:47.755078 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:47.755045 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-4wrx5/must-gather-zprmd"] Jul 07 12:05:47.755570 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:47.755239 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-4wrx5/must-gather-zprmd" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="copy" containerID="cri-o://5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79" gracePeriod=2 Jul 07 12:05:47.757572 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:47.757546 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-4wrx5/must-gather-zprmd"] Jul 07 12:05:47.757887 ip-10-0-138-138 kubenswrapper[2580]: E0707 12:05:47.757862 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-zprmd\" is forbidden: User \"system:node:ip-10-0-138-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4wrx5\": no relationship found between node 'ip-10-0-138-138.ec2.internal' and this object" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:47.977037 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:47.977013 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4wrx5_must-gather-zprmd_ad93885a-3abd-45a0-9f14-fed7259cef28/copy/0.log" Jul 07 12:05:47.979591 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:47.979575 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:48.071427 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.071358 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ad93885a-3abd-45a0-9f14-fed7259cef28-must-gather-output\") pod \"ad93885a-3abd-45a0-9f14-fed7259cef28\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " Jul 07 12:05:48.071522 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.071429 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfmn9\" (UniqueName: \"kubernetes.io/projected/ad93885a-3abd-45a0-9f14-fed7259cef28-kube-api-access-mfmn9\") pod \"ad93885a-3abd-45a0-9f14-fed7259cef28\" (UID: \"ad93885a-3abd-45a0-9f14-fed7259cef28\") " Jul 07 12:05:48.072607 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.072578 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad93885a-3abd-45a0-9f14-fed7259cef28-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "ad93885a-3abd-45a0-9f14-fed7259cef28" (UID: "ad93885a-3abd-45a0-9f14-fed7259cef28"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:05:48.073469 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.073446 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad93885a-3abd-45a0-9f14-fed7259cef28-kube-api-access-mfmn9" (OuterVolumeSpecName: "kube-api-access-mfmn9") pod "ad93885a-3abd-45a0-9f14-fed7259cef28" (UID: "ad93885a-3abd-45a0-9f14-fed7259cef28"). InnerVolumeSpecName "kube-api-access-mfmn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:05:48.172167 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.172142 2580 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ad93885a-3abd-45a0-9f14-fed7259cef28-must-gather-output\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 12:05:48.172167 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.172163 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfmn9\" (UniqueName: \"kubernetes.io/projected/ad93885a-3abd-45a0-9f14-fed7259cef28-kube-api-access-mfmn9\") on node \"ip-10-0-138-138.ec2.internal\" DevicePath \"\"" Jul 07 12:05:48.491604 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.491577 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4wrx5_must-gather-zprmd_ad93885a-3abd-45a0-9f14-fed7259cef28/copy/0.log" Jul 07 12:05:48.491950 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.491924 2580 generic.go:356] "Generic (PLEG): container finished" podID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerID="5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79" exitCode=143 Jul 07 12:05:48.492004 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.491981 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wrx5/must-gather-zprmd" Jul 07 12:05:48.492040 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.492002 2580 scope.go:117] "RemoveContainer" containerID="5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79" Jul 07 12:05:48.501109 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.501002 2580 scope.go:117] "RemoveContainer" containerID="11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09" Jul 07 12:05:48.513460 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.513442 2580 scope.go:117] "RemoveContainer" containerID="5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79" Jul 07 12:05:48.513711 ip-10-0-138-138 kubenswrapper[2580]: E0707 12:05:48.513693 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79\": container with ID starting with 5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79 not found: ID does not exist" containerID="5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79" Jul 07 12:05:48.513758 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.513719 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79"} err="failed to get container status \"5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79\": rpc error: code = NotFound desc = could not find container \"5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79\": container with ID starting with 5e201340f9a291a7ccd0f004f6ac5a4cd3a1bfda08a5e249d2b8774d6a2d4e79 not found: ID does not exist" Jul 07 12:05:48.513758 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.513736 2580 scope.go:117] "RemoveContainer" containerID="11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09" Jul 07 12:05:48.513935 ip-10-0-138-138 kubenswrapper[2580]: E0707 12:05:48.513920 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09\": container with ID starting with 11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09 not found: ID does not exist" containerID="11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09" Jul 07 12:05:48.513979 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:48.513939 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09"} err="failed to get container status \"11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09\": rpc error: code = NotFound desc = could not find container \"11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09\": container with ID starting with 11868ddedcb56a07289b53abe1f6ba75cf8fd08d8827cf607b969c6275377d09 not found: ID does not exist" Jul 07 12:05:49.486723 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.486691 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8dbvs_eb8ca7c0-60ef-453f-81c7-77dfe9000b45/kube-state-metrics/0.log" Jul 07 12:05:49.507206 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.507172 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8dbvs_eb8ca7c0-60ef-453f-81c7-77dfe9000b45/kube-rbac-proxy-main/0.log" Jul 07 12:05:49.539446 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.539421 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8dbvs_eb8ca7c0-60ef-453f-81c7-77dfe9000b45/kube-rbac-proxy-self/0.log" Jul 07 12:05:49.604589 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.604562 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-g8t4z_624086bc-f466-4257-b73b-d514b06fd259/monitoring-plugin/0.log" Jul 07 12:05:49.803381 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.803313 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nrbp2_3cadb9d6-22ba-49c5-9aaf-95752c1a25a8/node-exporter/0.log" Jul 07 12:05:49.833425 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.833399 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" path="/var/lib/kubelet/pods/ad93885a-3abd-45a0-9f14-fed7259cef28/volumes" Jul 07 12:05:49.841384 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.841365 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nrbp2_3cadb9d6-22ba-49c5-9aaf-95752c1a25a8/kube-rbac-proxy/0.log" Jul 07 12:05:49.885344 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:49.885315 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nrbp2_3cadb9d6-22ba-49c5-9aaf-95752c1a25a8/init-textfile/0.log" Jul 07 12:05:50.147435 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.147411 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/prometheus/0.log" Jul 07 12:05:50.187453 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.187433 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/config-reloader/0.log" Jul 07 12:05:50.224957 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.224938 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/thanos-sidecar/0.log" Jul 07 12:05:50.254663 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.254643 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/kube-rbac-proxy-web/0.log" Jul 07 12:05:50.281751 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.281732 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/kube-rbac-proxy/0.log" Jul 07 12:05:50.304524 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.304501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/kube-rbac-proxy-thanos/0.log" Jul 07 12:05:50.329161 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.329138 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_22f3b1cd-2b3d-4800-bb9b-5ee2337a880a/init-config-reloader/0.log" Jul 07 12:05:50.360550 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.360533 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-6hh2x_91c6f39d-d6b0-4449-8857-46ed8bf3c45b/prometheus-operator/0.log" Jul 07 12:05:50.387905 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:50.387877 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-6hh2x_91c6f39d-d6b0-4449-8857-46ed8bf3c45b/kube-rbac-proxy/0.log" Jul 07 12:05:53.133601 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:53.133567 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-tdg97_de8b8d4e-3667-4306-80d3-66b26df43a1b/download-server/0.log" Jul 07 12:05:54.725683 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.725641 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kc454_8ebddf9a-a072-4c3c-a78e-047ae767d096/dns/0.log" Jul 07 12:05:54.749154 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.749118 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kc454_8ebddf9a-a072-4c3c-a78e-047ae767d096/kube-rbac-proxy/0.log" Jul 07 12:05:54.851662 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.851638 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rpxgg_55207ce8-d0ca-4d60-a966-daa4e9ecca81/dns-node-resolver/0.log" Jul 07 12:05:54.950719 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.950689 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w"] Jul 07 12:05:54.950989 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.950975 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="copy" Jul 07 12:05:54.951033 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.950992 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="copy" Jul 07 12:05:54.951033 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.951004 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="gather" Jul 07 12:05:54.951033 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.951010 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="gather" Jul 07 12:05:54.951126 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.951050 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="gather" Jul 07 12:05:54.951126 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.951059 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad93885a-3abd-45a0-9f14-fed7259cef28" containerName="copy" Jul 07 12:05:54.956189 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.956170 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:54.958261 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.958232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n47qj\"/\"default-dockercfg-jsv9v\"" Jul 07 12:05:54.958367 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.958350 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n47qj\"/\"openshift-service-ca.crt\"" Jul 07 12:05:54.958644 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.958626 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n47qj\"/\"kube-root-ca.crt\"" Jul 07 12:05:54.962360 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:54.962339 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w"] Jul 07 12:05:55.030968 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.030909 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2sb9\" (UniqueName: \"kubernetes.io/projected/01171021-8a1b-4962-807a-14b046c094e1-kube-api-access-m2sb9\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.030968 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.030944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-podres\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.031092 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.030976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-proc\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.031092 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.031013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-lib-modules\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.031092 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.031053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-sys\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.131946 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.131913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-sys\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132095 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.131962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2sb9\" (UniqueName: \"kubernetes.io/projected/01171021-8a1b-4962-807a-14b046c094e1-kube-api-access-m2sb9\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132095 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.131987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-podres\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132095 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.132029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-proc\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132095 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.132061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-lib-modules\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132095 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.132030 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-sys\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132361 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.132113 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-proc\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132361 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.132114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-podres\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.132361 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.132254 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01171021-8a1b-4962-807a-14b046c094e1-lib-modules\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.140177 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.140157 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2sb9\" (UniqueName: \"kubernetes.io/projected/01171021-8a1b-4962-807a-14b046c094e1-kube-api-access-m2sb9\") pod \"perf-node-gather-daemonset-62t5w\" (UID: \"01171021-8a1b-4962-807a-14b046c094e1\") " pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.266963 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.266924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.387671 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.387646 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7759c787bb-wkb9p_6fe1dc1c-d412-4e29-8a50-75e632665d6a/registry/0.log" Jul 07 12:05:55.389946 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.389924 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w"] Jul 07 12:05:55.392559 ip-10-0-138-138 kubenswrapper[2580]: W0707 12:05:55.392530 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod01171021_8a1b_4962_807a_14b046c094e1.slice/crio-41b5a9ff0768cf7c408d3574281bb2487144f621961904c422828bc0083652fe WatchSource:0}: Error finding container 41b5a9ff0768cf7c408d3574281bb2487144f621961904c422828bc0083652fe: Status 404 returned error can't find the container with id 41b5a9ff0768cf7c408d3574281bb2487144f621961904c422828bc0083652fe Jul 07 12:05:55.459177 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.459159 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ztk6q_b9d58c15-95ca-4172-a2ab-c76893c6ebea/node-ca/0.log" Jul 07 12:05:55.519180 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.519155 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" event={"ID":"01171021-8a1b-4962-807a-14b046c094e1","Type":"ContainerStarted","Data":"c0d9c0065581ab9af3339427ea23a2731deba5c9a3cfd0d5f115cfaa9ddcd23a"} Jul 07 12:05:55.519305 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.519212 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" event={"ID":"01171021-8a1b-4962-807a-14b046c094e1","Type":"ContainerStarted","Data":"41b5a9ff0768cf7c408d3574281bb2487144f621961904c422828bc0083652fe"} Jul 07 12:05:55.519305 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.519241 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:05:55.535750 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:55.535709 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" podStartSLOduration=1.5356948259999998 podStartE2EDuration="1.535694826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:05:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:05:55.533768028 +0000 UTC m=+1544.377492128" watchObservedRunningTime="2026-07-07 12:05:55.535694826 +0000 UTC m=+1544.379418925" Jul 07 12:05:56.659143 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:56.659106 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wcrp5_4aac21db-70cd-4e10-b7ff-700f56b241ba/serve-healthcheck-canary/0.log" Jul 07 12:05:57.231680 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:57.231634 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlxns_6583d603-c819-4182-a976-595b8fa41475/kube-rbac-proxy/0.log" Jul 07 12:05:57.251523 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:57.251487 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlxns_6583d603-c819-4182-a976-595b8fa41475/exporter/0.log" Jul 07 12:05:57.274101 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:57.274079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlxns_6583d603-c819-4182-a976-595b8fa41475/extractor/0.log" Jul 07 12:05:59.467574 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:59.467544 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-dqnnw_8ab92d5f-30d1-4e02-99e5-b34210e2ce1d/s3-init/0.log" Jul 07 12:05:59.482504 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:05:59.482478 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-dqnnw_8ab92d5f-30d1-4e02-99e5-b34210e2ce1d/download-hf-model/0.log" Jul 07 12:06:01.532935 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:01.532909 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n47qj/perf-node-gather-daemonset-62t5w" Jul 07 12:06:03.137033 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:03.137006 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-tlbfl_17b3e0f5-3744-471c-83a6-13588da0fdb8/migrator/0.log" Jul 07 12:06:03.155888 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:03.155863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-tlbfl_17b3e0f5-3744-471c-83a6-13588da0fdb8/graceful-termination/0.log" Jul 07 12:06:04.724897 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.724873 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/kube-multus-additional-cni-plugins/0.log" Jul 07 12:06:04.743825 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.743789 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/egress-router-binary-copy/0.log" Jul 07 12:06:04.766245 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.766187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/cni-plugins/0.log" Jul 07 12:06:04.784689 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.784667 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/bond-cni-plugin/0.log" Jul 07 12:06:04.807081 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.807063 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/routeoverride-cni/0.log" Jul 07 12:06:04.827282 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.827263 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/whereabouts-cni-bincopy/0.log" Jul 07 12:06:04.847308 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:04.847287 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cgbd2_b5c6c0cc-9fe5-4237-a5a5-2156991897d5/whereabouts-cni/0.log" Jul 07 12:06:05.088487 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:05.088388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gqrt9_b5f90c71-15d1-435d-82a2-dec60a9655f5/kube-multus/0.log" Jul 07 12:06:05.137212 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:05.137180 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2ml8h_529387f7-20c3-4ce0-aa48-3861845232aa/network-metrics-daemon/0.log" Jul 07 12:06:05.154178 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:05.154158 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2ml8h_529387f7-20c3-4ce0-aa48-3861845232aa/kube-rbac-proxy/0.log" Jul 07 12:06:06.020568 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.020502 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/ovn-controller/0.log" Jul 07 12:06:06.044310 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.044289 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/ovn-acl-logging/0.log" Jul 07 12:06:06.073896 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.073876 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/kube-rbac-proxy-node/0.log" Jul 07 12:06:06.098618 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.098596 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/kube-rbac-proxy-ovn-metrics/0.log" Jul 07 12:06:06.128598 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.128576 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/northd/0.log" Jul 07 12:06:06.153865 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.153850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/nbdb/0.log" Jul 07 12:06:06.175036 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.175017 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/sbdb/0.log" Jul 07 12:06:06.215273 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:06.215252 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2c7qq_2375073f-e7ec-4841-9d69-cf8a98623320/ovnkube-controller/0.log" Jul 07 12:06:07.722043 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:07.722013 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qdr6z_f0a98300-a855-4d77-a89c-100c5e06e501/network-check-target-container/0.log" Jul 07 12:06:08.602795 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:08.602771 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-7gww9_28111496-14cc-4e90-ae0c-88d61f489e9b/iptables-alerter/0.log" Jul 07 12:06:09.250014 ip-10-0-138-138 kubenswrapper[2580]: I0707 12:06:09.249988 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-2d92w_e2e5b17d-5bd6-4eb1-8b04-d331500eab5d/tuned/0.log"