Jul 16 17:33:37.386307 ip-10-0-141-151 systemd[1]: Starting Kubernetes Kubelet... Jul 16 17:33:37.839650 ip-10-0-141-151 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:33:37.839650 ip-10-0-141-151 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 17:33:37.839650 ip-10-0-141-151 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:33:37.839650 ip-10-0-141-151 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 17:33:37.839650 ip-10-0-141-151 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:33:37.840776 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.840669 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 17:33:37.847898 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847877 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:33:37.847898 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847893 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:33:37.847898 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847898 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:33:37.847898 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847901 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:33:37.847898 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847904 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:33:37.847898 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847908 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847911 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847914 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847917 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847919 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847922 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847925 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847928 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847930 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847933 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847935 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847938 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847940 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847943 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847945 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847947 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847950 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847952 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847955 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847957 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:33:37.848128 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847960 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847962 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847965 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847967 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847970 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847972 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847975 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847977 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847979 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847982 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847984 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847986 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847988 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847991 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847994 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847997 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.847999 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848002 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848005 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:33:37.848605 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848008 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848010 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848012 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848014 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848017 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848019 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848040 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848044 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848046 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848049 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848051 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848053 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848057 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848059 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848061 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848064 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848066 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848069 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848071 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848073 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:33:37.849053 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848076 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848081 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848084 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848087 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848090 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848092 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848095 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848098 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848101 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848103 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848106 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848108 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848110 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848113 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848115 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848119 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848121 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848124 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848126 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848128 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:33:37.849542 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848131 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848134 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848136 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848139 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848142 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848144 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848148 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848152 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848156 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848159 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848162 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848164 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848167 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848169 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848172 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848879 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848886 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848891 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:33:37.850005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848895 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848899 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848903 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848906 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848909 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848911 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848914 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848916 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848919 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848921 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848924 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848927 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848929 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848931 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848934 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848937 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848939 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848942 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848944 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:33:37.850471 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848946 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848949 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848951 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848954 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848956 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848959 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848961 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848963 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848966 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848968 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848971 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848973 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848975 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848978 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848982 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848984 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848987 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848989 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848992 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848994 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:33:37.850912 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848996 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.848998 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849001 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849003 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849006 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849009 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849011 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849014 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849016 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849019 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849040 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849045 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849048 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849050 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849053 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849057 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849060 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849064 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849067 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849070 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:33:37.851428 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849073 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849076 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849078 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849081 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849083 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849086 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849089 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849092 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849094 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849096 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849099 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849101 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849104 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849106 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849108 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849111 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849113 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849115 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849118 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849120 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:33:37.851905 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849122 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849126 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849128 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849131 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849133 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849135 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849138 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849140 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849142 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849145 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849147 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849150 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849152 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849155 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849157 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849160 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849163 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849522 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849527 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849530 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:33:37.852396 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849533 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849536 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849539 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849541 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849544 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849547 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849549 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849552 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849555 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849558 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849560 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849563 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849565 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849569 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849571 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849574 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849576 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849579 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849582 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849584 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:33:37.852929 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849587 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849589 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849592 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849594 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849597 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849599 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849602 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849604 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849606 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849608 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849611 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849613 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849615 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849618 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849620 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849622 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849625 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849627 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849629 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849631 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:33:37.853412 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849633 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849636 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849638 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849641 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849645 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849648 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849651 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849654 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849659 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849662 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849664 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849667 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849669 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849672 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849674 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849677 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849679 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849681 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849684 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849687 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:33:37.853879 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849689 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849691 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849694 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849696 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849699 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849701 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849704 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849706 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849708 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849711 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849713 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849715 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849719 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849723 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849727 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849730 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849733 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849735 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849738 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:33:37.854370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849741 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849745 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849749 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849751 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849754 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849756 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849760 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849763 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849765 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849767 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849770 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849772 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849774 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849777 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849779 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849782 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.849784 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849847 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849854 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849860 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849864 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 17:33:37.854808 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849869 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849874 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849880 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849885 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849888 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849891 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849894 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849897 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849900 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849903 2579 flags.go:64] FLAG: --cgroup-root="" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849905 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849908 2579 flags.go:64] FLAG: --client-ca-file="" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849913 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849916 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849922 2579 flags.go:64] FLAG: --cluster-domain="" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849925 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849928 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849930 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849934 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849938 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849940 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849943 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849946 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849949 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 16 17:33:37.855304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849952 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849955 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849957 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849960 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849964 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849967 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849969 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849972 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849975 2579 flags.go:64] FLAG: --enable-server="true" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849977 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849981 2579 flags.go:64] FLAG: --event-burst="100" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849984 2579 flags.go:64] FLAG: --event-qps="50" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849987 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849989 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849992 2579 flags.go:64] FLAG: --eviction-hard="" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849995 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.849999 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850002 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850005 2579 flags.go:64] FLAG: --eviction-soft="" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850008 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850012 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850016 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850019 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850040 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850043 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 17:33:37.855860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850046 2579 flags.go:64] FLAG: --feature-gates="" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850050 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850053 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850056 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850059 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850061 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850064 2579 flags.go:64] FLAG: --help="false" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850067 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-141-151.ec2.internal" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850069 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850072 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850075 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850078 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850081 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850084 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850086 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850089 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850092 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850095 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850098 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850100 2579 flags.go:64] FLAG: --kube-reserved="" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850103 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850106 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850109 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850112 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 17:33:37.856473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850114 2579 flags.go:64] FLAG: --lock-file="" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850117 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850120 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850123 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850128 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850131 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850134 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850136 2579 flags.go:64] FLAG: --logging-format="text" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850139 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850142 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850145 2579 flags.go:64] FLAG: --manifest-url="" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850148 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850152 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850155 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850159 2579 flags.go:64] FLAG: --max-pods="110" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850161 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850164 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850167 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850169 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850173 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850175 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850178 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850184 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850187 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850190 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 17:33:37.857087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850192 2579 flags.go:64] FLAG: --pod-cidr="" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850195 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850200 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850203 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850206 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850208 2579 flags.go:64] FLAG: --port="10250" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850211 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850214 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-037c0e59913f12bdb" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850217 2579 flags.go:64] FLAG: --qos-reserved="" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850219 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850222 2579 flags.go:64] FLAG: --register-node="true" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850225 2579 flags.go:64] FLAG: --register-with-taints="" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850228 2579 flags.go:64] FLAG: --registry-burst="10" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850231 2579 flags.go:64] FLAG: --registry-qps="5" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850234 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850237 2579 flags.go:64] FLAG: --reserved-memory="" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850240 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850243 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850246 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850248 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850251 2579 flags.go:64] FLAG: --runonce="false" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850253 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850256 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850259 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850261 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 17:33:37.857704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850264 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850266 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850269 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850272 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850274 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850277 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850280 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850282 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850285 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850288 2579 flags.go:64] FLAG: --system-cgroups="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850290 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850295 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850298 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850300 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850304 2579 flags.go:64] FLAG: --tls-min-version="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850307 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850309 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850312 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850315 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850317 2579 flags.go:64] FLAG: --v="2" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850321 2579 flags.go:64] FLAG: --version="false" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850325 2579 flags.go:64] FLAG: --vmodule="" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850329 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.850332 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850449 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:33:37.858309 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850452 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850455 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850458 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850460 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850462 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850465 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850467 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850470 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850473 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850475 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850478 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850481 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850485 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850488 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850491 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850493 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850496 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850498 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850500 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850503 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:33:37.858886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850505 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850508 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850510 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850513 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850515 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850517 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850520 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850522 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850526 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850530 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850533 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850535 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850538 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850540 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850542 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850544 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850547 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850549 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850552 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:33:37.859370 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850554 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850556 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850559 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850561 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850563 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850565 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850568 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850570 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850572 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850575 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850577 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850579 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850582 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850586 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850588 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850590 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850593 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850595 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850600 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850602 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:33:37.859858 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850605 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850607 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850610 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850613 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850615 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850618 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850620 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850623 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850625 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850628 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850630 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850633 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850635 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850637 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850639 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850642 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850645 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850647 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850650 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:33:37.860346 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850652 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850655 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850657 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850660 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850662 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850664 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850670 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850672 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850674 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850678 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850681 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850686 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850689 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850691 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850694 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850697 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850699 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850703 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850706 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:33:37.861564 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.850708 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.851321 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.858588 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.858603 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.858622 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.858627 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858681 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858686 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858689 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858692 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858695 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858698 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858701 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858705 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:33:37.862003 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858709 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858712 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858715 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858718 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858720 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858723 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858725 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858728 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858730 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858732 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858735 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858737 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858740 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858743 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858747 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858750 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858752 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858755 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858758 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858760 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:33:37.862358 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858763 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858767 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858769 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858772 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858774 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858777 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858779 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858782 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858785 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858787 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858789 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858792 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858794 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858796 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858799 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858801 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858803 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858806 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858808 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858811 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:33:37.862823 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858813 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858815 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858818 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858820 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858823 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858825 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858827 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858830 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858832 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858834 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858836 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858839 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858841 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858844 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858847 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858850 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858853 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858855 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858858 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:33:37.863359 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858860 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858863 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858865 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858868 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858870 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858874 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858877 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858880 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858883 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858887 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858890 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858893 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858895 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858898 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858902 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858904 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858907 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858910 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858912 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858914 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:33:37.863842 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858917 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858919 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858921 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858924 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858927 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858930 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858933 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858936 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858939 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858941 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858944 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.858946 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.858951 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859077 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:33:37.864322 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859082 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859085 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859088 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859091 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859093 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859096 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859098 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859101 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859103 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859106 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859108 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859111 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859113 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859116 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859118 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859121 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859123 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859127 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859131 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:33:37.864660 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859134 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859137 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859139 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859142 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859144 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859147 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859149 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859152 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859155 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859158 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859161 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859163 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859166 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859168 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859170 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859173 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859175 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859177 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859180 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859182 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:33:37.865205 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859184 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859187 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859189 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859192 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859194 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859197 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859199 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859202 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859204 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859206 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859208 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859211 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859213 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859216 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859220 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859222 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859225 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859228 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859230 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:33:37.865671 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859233 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859236 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859239 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859241 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859244 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859247 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859249 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859251 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859254 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859256 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859258 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859260 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859263 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859265 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859267 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859270 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859273 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859276 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859278 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859280 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:33:37.866213 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859283 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859285 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859287 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859290 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859292 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859294 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859297 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859299 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859301 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859304 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859306 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859308 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859310 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859313 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859315 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859318 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859320 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859323 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859325 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:33:37.866681 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:37.859328 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:33:37.867404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.859332 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:33:37.867404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.859432 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 17:33:37.867404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.862807 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 17:33:37.867404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.864592 2579 server.go:1019] "Starting client certificate rotation" Jul 16 17:33:37.867404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.864709 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 17:33:37.867404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.864757 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 17:33:37.867656 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.867448 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 17:33:37.868619 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.868597 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 17:33:37.876874 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.876857 2579 log.go:25] "Validated CRI v1 runtime API" Jul 16 17:33:37.879360 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.879345 2579 log.go:25] "Validated CRI v1 image API" Jul 16 17:33:37.880178 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.880166 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 17:33:37.882685 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.882668 2579 fs.go:135] Filesystem UUIDs: map[4a61a855-6c79-45c6-9756-19312692e83b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bce376dc-e9c7-4b80-a1a6-1b46346fab15:/dev/nvme0n1p4] Jul 16 17:33:37.882741 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.882685 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 17:33:37.887548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.887440 2579 manager.go:216] Machine: {Timestamp:2026-07-16 17:33:37.886219495 +0000 UTC m=+0.372512502 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3250210 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24a103911b1fbc9c5470bf74cb6a56 SystemUUID:ec24a103-911b-1fbc-9c54-70bf74cb6a56 BootID:fddc6bc3-3890-4567-94b9-9984c053063e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:af:b7:fd:e3:f3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:af:b7:fd:e3:f3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:af:a2:04:04:e1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 17:33:37.887548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.887543 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 17:33:37.887673 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.887615 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 17:33:37.888678 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.888653 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 17:33:37.888801 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.888679 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-151.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 17:33:37.889508 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.889498 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 17:33:37.889542 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.889511 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 17:33:37.889542 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.889518 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 17:33:37.889542 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.889531 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 17:33:37.891520 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.891508 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 17:33:37.896370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896358 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 16 17:33:37.896476 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896467 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 17:33:37.896508 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896499 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 16 17:33:37.896537 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896509 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 17:33:37.896537 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896534 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 17:33:37.896621 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896542 2579 kubelet.go:398] "Adding apiserver pod source" Jul 16 17:33:37.896621 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896556 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 17:33:37.896896 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896875 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 17:33:37.896896 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.896894 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 17:33:37.899040 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.899005 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 17:33:37.900714 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.900697 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 17:33:37.903011 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.902994 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 17:33:37.903102 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903050 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 17:33:37.903240 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903227 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 17:33:37.903296 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903247 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 17:33:37.903296 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903257 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 17:33:37.903296 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903265 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 17:33:37.903296 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903277 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 17:33:37.903296 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903284 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 17:33:37.903885 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903874 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 17:33:37.903935 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903886 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 17:33:37.903935 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903896 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 17:33:37.903935 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903905 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 17:33:37.903935 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903924 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 17:33:37.904068 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.903936 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 17:33:37.904711 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.904699 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 17:33:37.904774 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.904714 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 17:33:37.909134 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.909105 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 17:33:37.909134 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.909121 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 17:33:37.909344 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.909328 2579 server.go:1268] "Started kubelet" Jul 16 17:33:37.910307 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.910246 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 17:33:37.910384 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.910366 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 17:33:37.910555 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.910494 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 17:33:37.910845 ip-10-0-141-151 systemd[1]: Started Kubernetes Kubelet. Jul 16 17:33:37.911645 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.911624 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 17:33:37.912120 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.912100 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-151.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 17:33:37.913145 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.913128 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 16 17:33:37.917394 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.917281 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 17:33:37.921859 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.920747 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-151.ec2.internal.18c2d5903f7abe58 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-151.ec2.internal,UID:ip-10-0-141-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-151.ec2.internal,},FirstTimestamp:2026-07-16 17:33:37.909153368 +0000 UTC m=+0.395446423,LastTimestamp:2026-07-16 17:33:37.909153368 +0000 UTC m=+0.395446423,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-151.ec2.internal,}" Jul 16 17:33:37.921946 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.921891 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 17:33:37.921991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.921956 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 17:33:37.922126 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.922107 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:37.922753 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922724 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 17:33:37.922753 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922730 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 17:33:37.922753 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922752 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 17:33:37.922930 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922870 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 16 17:33:37.922930 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922880 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 16 17:33:37.922930 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922880 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 17:33:37.922930 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922902 2579 factory.go:55] Registering systemd factory Jul 16 17:33:37.922930 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.922913 2579 factory.go:223] Registration of the systemd container factory successfully Jul 16 17:33:37.923245 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.923231 2579 factory.go:153] Registering CRI-O factory Jul 16 17:33:37.923245 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.923246 2579 factory.go:223] Registration of the crio container factory successfully Jul 16 17:33:37.923389 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.923270 2579 factory.go:103] Registering Raw factory Jul 16 17:33:37.923389 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.923284 2579 manager.go:1196] Started watching for new ooms in manager Jul 16 17:33:37.924365 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.924338 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 17:33:37.924542 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.924469 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 17:33:37.924542 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.924523 2579 manager.go:349] Starting recovery of all containers Jul 16 17:33:37.934713 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.934696 2579 manager.go:354] Recovery completed Jul 16 17:33:37.939151 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.939139 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:37.940952 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.940934 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:37.941014 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.940968 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:37.941014 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.940980 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:37.941474 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.941459 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 17:33:37.941474 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.941474 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 17:33:37.941591 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.941491 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 16 17:33:37.941591 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.941505 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:37.942157 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.942140 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:37.942231 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.942167 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:37.942231 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.942176 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:37.943726 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.943704 2579 policy_none.go:49] "None policy: Start" Jul 16 17:33:37.943790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.943740 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 17:33:37.943790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.943755 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 17:33:37.945705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.945692 2579 policy_none.go:47] "Start" Jul 16 17:33:37.950227 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.950075 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-856g2" Jul 16 17:33:37.952106 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.952010 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-151.ec2.internal.18c2d590415ff2cf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-151.ec2.internal,UID:ip-10-0-141-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-151.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-151.ec2.internal,},FirstTimestamp:2026-07-16 17:33:37.940951759 +0000 UTC m=+0.427244754,LastTimestamp:2026-07-16 17:33:37.940951759 +0000 UTC m=+0.427244754,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-151.ec2.internal,}" Jul 16 17:33:37.955576 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.955559 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-856g2" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987485 2579 manager.go:337] "Starting Device Plugin manager" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.987513 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987523 2579 server.go:87] "Starting device plugin registration server" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987751 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987762 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987863 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987937 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:37.987945 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.988490 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 17:33:38.001965 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:37.988531 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.056904 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.056864 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 17:33:38.058177 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.058155 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 17:33:38.058268 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.058189 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 17:33:38.058268 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.058214 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 17:33:38.058268 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.058252 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 17:33:38.060832 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.060814 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:33:38.088237 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.088220 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:38.088995 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.088980 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:38.089076 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.089008 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:38.089076 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.089018 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:38.089076 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.089061 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.098543 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.098505 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.098543 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.098528 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-151.ec2.internal\": node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.114978 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.114961 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.159323 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.159292 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal"] Jul 16 17:33:38.159439 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.159362 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:38.160672 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.160657 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:38.160750 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.160689 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:38.160750 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.160703 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:38.162935 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.162912 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:38.163039 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163016 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.163097 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163055 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:38.163622 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163606 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:38.163695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163638 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:38.163695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163609 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:38.163695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163671 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:38.163695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163651 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:38.163822 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.163692 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:38.165829 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.165811 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.165916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.165840 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:33:38.166462 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.166448 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:33:38.166541 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.166475 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:33:38.166541 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.166488 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:33:38.187527 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.187508 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-151.ec2.internal\" not found" node="ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.191734 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.191717 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-151.ec2.internal\" not found" node="ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.215466 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.215445 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.224548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.224526 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cd1a81f81de33839e1fd7043ad57b780-config\") pod \"kube-apiserver-proxy-ip-10-0-141-151.ec2.internal\" (UID: \"cd1a81f81de33839e1fd7043ad57b780\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.224629 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.224553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d13bff1d03965b595bba576b1d6f637-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal\" (UID: \"5d13bff1d03965b595bba576b1d6f637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.224629 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.224580 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d13bff1d03965b595bba576b1d6f637-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal\" (UID: \"5d13bff1d03965b595bba576b1d6f637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.316282 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.316261 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.325658 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.325635 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d13bff1d03965b595bba576b1d6f637-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal\" (UID: \"5d13bff1d03965b595bba576b1d6f637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.325735 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.325670 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d13bff1d03965b595bba576b1d6f637-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal\" (UID: \"5d13bff1d03965b595bba576b1d6f637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.325735 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.325701 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cd1a81f81de33839e1fd7043ad57b780-config\") pod \"kube-apiserver-proxy-ip-10-0-141-151.ec2.internal\" (UID: \"cd1a81f81de33839e1fd7043ad57b780\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.325735 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.325701 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d13bff1d03965b595bba576b1d6f637-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal\" (UID: \"5d13bff1d03965b595bba576b1d6f637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.325866 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.325743 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d13bff1d03965b595bba576b1d6f637-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal\" (UID: \"5d13bff1d03965b595bba576b1d6f637\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.325866 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.325803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cd1a81f81de33839e1fd7043ad57b780-config\") pod \"kube-apiserver-proxy-ip-10-0-141-151.ec2.internal\" (UID: \"cd1a81f81de33839e1fd7043ad57b780\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.417132 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.417054 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.489530 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.489501 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.495899 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.495883 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.517884 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.517863 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.618432 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.618399 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.719040 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.718958 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.819460 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.819433 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-151.ec2.internal\" not found" Jul 16 17:33:38.863952 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.863925 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 17:33:38.864490 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.864104 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 17:33:38.888417 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.888394 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:33:38.897546 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.897529 2579 apiserver.go:52] "Watching apiserver" Jul 16 17:33:38.909032 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.909002 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 17:33:38.909273 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.909252 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-j4w9x","openshift-dns/node-resolver-72cjz","openshift-image-registry/node-ca-6rsl6","openshift-network-diagnostics/network-check-target-b87sw","openshift-ovn-kubernetes/ovnkube-node-jcjq6","kube-system/global-pull-secret-syncer-4tq6n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t","openshift-multus/multus-additional-cni-plugins-ncx2q","openshift-multus/multus-h8kcl","openshift-multus/network-metrics-daemon-nm4p2","openshift-network-operator/iptables-alerter-tcbhq","kube-system/konnectivity-agent-2mbhd"] Jul 16 17:33:38.911802 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.911788 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.913794 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.913776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:38.914271 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.914250 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cnvkw\"" Jul 16 17:33:38.914429 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.914410 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.914735 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.914719 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.915805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.915790 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:38.916197 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.916181 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wfgd5\"" Jul 16 17:33:38.916343 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.916326 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.916403 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.916349 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.917916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.917902 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:38.917997 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.917968 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:38.918341 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.918322 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.918412 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.918343 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r6mcv\"" Jul 16 17:33:38.918412 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.918370 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.918695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.918680 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 17:33:38.920044 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.920017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.922087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922071 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 17:33:38.922179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922134 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 17:33:38.922179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922158 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.922335 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922325 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.922384 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922368 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 17:33:38.922693 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922679 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h525p\"" Jul 16 17:33:38.922953 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922933 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 17:33:38.923017 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922951 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 17:33:38.923017 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922940 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.923017 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.922987 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.924698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.924681 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l84nq\"" Jul 16 17:33:38.924698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.924695 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.924820 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.924738 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 17:33:38.924992 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.924976 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.924992 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.925000 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.927194 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.927168 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h8kcl" Jul 16 17:33:38.927369 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.927355 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 17:33:38.927593 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.927577 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.927829 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.927815 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mjnrc\"" Jul 16 17:33:38.927909 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.927895 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 17:33:38.927956 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.927928 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 17:33:38.928189 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.928176 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.929234 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-kubelet\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929318 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929246 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-registration-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.929318 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929281 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a335436-5134-4780-95ff-b3d182fb2969-host\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:38.929318 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-systemd-units\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929318 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929311 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-tuned\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929330 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-system-cni-dir\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929373 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-sys\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929400 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-modprobe-d\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929424 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-systemd\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929444 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-run\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929468 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-os-release\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929489 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.929528 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-var-lib-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929553 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-node-log\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.929608 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e74d041-c34c-4f23-8952-a747e8c312a2-ovn-node-metrics-cert\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-log-socket\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-ovn\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.929867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929714 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9vd4k\"" Jul 16 17:33:38.930231 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.929994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-cni-netd\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.930231 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930055 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-var-lib-kubelet\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.930231 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-sys-fs\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.930231 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gfn5\" (UniqueName: \"kubernetes.io/projected/04283751-84b7-4886-8a2e-e7e3085d2377-kube-api-access-2gfn5\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.930397 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930252 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.930397 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930312 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-ovnkube-config\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.930397 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930343 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-cnibin\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.930397 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98m5j\" (UniqueName: \"kubernetes.io/projected/1a335436-5134-4780-95ff-b3d182fb2969-kube-api-access-98m5j\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:38.930518 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-kubernetes\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.930675 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysctl-d\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.930717 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930695 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-etc-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.930747 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-lib-modules\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.930799 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.930920 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysctl-conf\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.930954 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-tmp-dir\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:38.931001 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.930988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-socket-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.931051 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-host\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.931116 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd1cb5-3def-4bae-9218-ceffe14067df-tmp\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.931149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931137 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-device-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.931195 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931180 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-slash\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931235 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hm8h\" (UniqueName: \"kubernetes.io/projected/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-kube-api-access-5hm8h\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:38.931327 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.931374 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931347 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pnp5\" (UniqueName: \"kubernetes.io/projected/e1c2402d-b45a-49ed-8f26-5cb19c6af258-kube-api-access-5pnp5\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.931418 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931393 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-ovnkube-script-lib\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931450 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931436 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbnq4\" (UniqueName: \"kubernetes.io/projected/8e74d041-c34c-4f23-8952-a747e8c312a2-kube-api-access-kbnq4\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931504 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931535 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-env-overrides\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931574 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.931605 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a335436-5134-4780-95ff-b3d182fb2969-serviceca\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:38.931635 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:38.931665 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-run-netns\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931718 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-hosts-file\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:38.931788 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-cni-binary-copy\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:38.931820 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-systemd\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931849 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931838 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-cni-bin\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:38.931880 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931869 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6pmd\" (UniqueName: \"kubernetes.io/projected/6ebd1cb5-3def-4bae-9218-ceffe14067df-kube-api-access-z6pmd\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.931917 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:38.931950 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.931940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysconfig\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:38.932411 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.932381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:38.932611 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.932573 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 17:33:38.932719 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.932706 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" Jul 16 17:33:38.935556 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.935538 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 17:33:38.935649 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.935629 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:33:38.935721 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.935695 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 17:33:38.935783 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.935752 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:38.935883 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:38.935863 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:38.936128 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.936088 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vj8sn\"" Jul 16 17:33:38.937822 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.937806 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal"] Jul 16 17:33:38.937896 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.937871 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:38.938762 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.938641 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 17:33:38.940542 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.940524 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 17:33:38.940654 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.940629 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4vpbd\"" Jul 16 17:33:38.940825 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.940813 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 17:33:38.948486 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.948472 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal"] Jul 16 17:33:38.948562 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.948550 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 17:33:38.960525 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.960475 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 17:28:38 +0000 UTC" deadline="2028-02-07 13:04:51.9044457 +0000 UTC" Jul 16 17:33:38.960525 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.960519 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13699h31m12.943929682s" Jul 16 17:33:38.967116 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.967100 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rmv5j" Jul 16 17:33:38.973457 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:38.973412 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rmv5j" Jul 16 17:33:39.024208 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.024185 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 17:33:39.032430 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032398 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pnp5\" (UniqueName: \"kubernetes.io/projected/e1c2402d-b45a-49ed-8f26-5cb19c6af258-kube-api-access-5pnp5\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-ovnkube-script-lib\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbnq4\" (UniqueName: \"kubernetes.io/projected/8e74d041-c34c-4f23-8952-a747e8c312a2-kube-api-access-kbnq4\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032486 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-env-overrides\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a335436-5134-4780-95ff-b3d182fb2969-serviceca\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-cni-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.032595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032595 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-cnibin\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032615 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-conf-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ghw2\" (UniqueName: \"kubernetes.io/projected/380ce908-8115-4e31-9216-4ffded74c95e-kube-api-access-4ghw2\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-run-netns\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-hosts-file\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032773 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032822 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.032898 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.032871 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-run-netns\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033128 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-env-overrides\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-cni-binary-copy\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.033408 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033389 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-hosts-file\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.033706 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033689 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x89dz\" (UniqueName: \"kubernetes.io/projected/84b3a525-e62d-44eb-b93f-2c8d1873420c-kube-api-access-x89dz\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:39.033798 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-systemd\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033798 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033766 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-cni-bin\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033798 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033779 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-ovnkube-script-lib\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033798 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6pmd\" (UniqueName: \"kubernetes.io/projected/6ebd1cb5-3def-4bae-9218-ceffe14067df-kube-api-access-z6pmd\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033799 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-systemd\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033814 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033838 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16e7f2ae-f5aa-4116-a508-eaf0787a813a-cni-binary-copy\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-cni-bin\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysconfig\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-kubelet\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-registration-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a335436-5134-4780-95ff-b3d182fb2969-host\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033951 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.033984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.033981 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-multus-certs\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-systemd-units\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034016 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysconfig\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-tuned\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034047 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-registration-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034082 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a335436-5134-4780-95ff-b3d182fb2969-serviceca\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034087 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-kubelet\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034111 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a335436-5134-4780-95ff-b3d182fb2969-host\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034117 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-systemd-units\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034118 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-system-cni-dir\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034087 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-system-cni-dir\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62952831-854a-48e2-886c-3cdf2492c97a-dbus\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-sys\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-system-cni-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-etc-kubernetes\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034223 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-modprobe-d\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034256 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-systemd\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034272 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-run\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.034459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034277 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-cni-binary-copy\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6j29\" (UniqueName: \"kubernetes.io/projected/16e7f2ae-f5aa-4116-a508-eaf0787a813a-kube-api-access-t6j29\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-os-release\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-var-lib-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-node-log\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034364 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-sys\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034387 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-run\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e74d041-c34c-4f23-8952-a747e8c312a2-ovn-node-metrics-cert\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-os-release\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034418 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-socket-dir-parent\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034463 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-modprobe-d\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-log-socket\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034531 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034541 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-systemd\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034601 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-log-socket\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034731 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.035300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-var-lib-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034875 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-node-log\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034922 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-os-release\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034941 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2dd25ad-b133-41cd-af46-3c7b865ea400-konnectivity-ca\") pod \"konnectivity-agent-2mbhd\" (UID: \"f2dd25ad-b133-41cd-af46-3c7b865ea400\") " pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.034969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-ovn\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035070 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-cni-netd\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035085 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-var-lib-kubelet\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035071 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-sys-fs\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-sys-fs\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035142 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gfn5\" (UniqueName: \"kubernetes.io/projected/04283751-84b7-4886-8a2e-e7e3085d2377-kube-api-access-2gfn5\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-ovn\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035171 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-hostroot\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-cni-netd\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035224 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-var-lib-kubelet\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035227 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-ovnkube-config\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-cnibin\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035295 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98m5j\" (UniqueName: \"kubernetes.io/projected/1a335436-5134-4780-95ff-b3d182fb2969-kube-api-access-98m5j\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035317 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035342 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-kubernetes\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035365 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysctl-d\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035386 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/04283751-84b7-4886-8a2e-e7e3085d2377-cnibin\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-etc-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035413 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-lib-modules\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035421 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035438 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-k8s-cni-cncf-io\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035462 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-netns\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035487 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62952831-854a-48e2-886c-3cdf2492c97a-kubelet-config\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035514 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035542 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysctl-conf\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-tmp-dir\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035602 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-socket-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-cni-multus\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.036938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-daemon-config\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/380ce908-8115-4e31-9216-4ffded74c95e-iptables-alerter-script\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-host\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd1cb5-3def-4bae-9218-ceffe14067df-tmp\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035774 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-device-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035797 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-cni-bin\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-slash\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035848 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hm8h\" (UniqueName: \"kubernetes.io/projected/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-kube-api-access-5hm8h\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2dd25ad-b133-41cd-af46-3c7b865ea400-agent-certs\") pod \"konnectivity-agent-2mbhd\" (UID: \"f2dd25ad-b133-41cd-af46-3c7b865ea400\") " pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035920 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-kubelet\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/380ce908-8115-4e31-9216-4ffded74c95e-host-slash\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036144 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-run-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036162 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysctl-d\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036202 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-host\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036214 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-kubernetes\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.037698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036241 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-etc-openvswitch\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.035824 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e74d041-c34c-4f23-8952-a747e8c312a2-ovnkube-config\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-lib-modules\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036242 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e74d041-c34c-4f23-8952-a747e8c312a2-host-slash\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036306 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-socket-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036437 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-sysctl-conf\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036437 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e1c2402d-b45a-49ed-8f26-5cb19c6af258-device-dir\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-tmp-dir\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.036853 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/04283751-84b7-4886-8a2e-e7e3085d2377-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.038340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd1cb5-3def-4bae-9218-ceffe14067df-etc-tuned\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.038367 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd1cb5-3def-4bae-9218-ceffe14067df-tmp\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.038548 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.038374 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e74d041-c34c-4f23-8952-a747e8c312a2-ovn-node-metrics-cert\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.040626 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.040605 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:33:39.040732 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.040634 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:33:39.040732 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.040647 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:39.040732 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.040713 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:33:39.540691448 +0000 UTC m=+2.026984480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:39.042762 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.042742 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pnp5\" (UniqueName: \"kubernetes.io/projected/e1c2402d-b45a-49ed-8f26-5cb19c6af258-kube-api-access-5pnp5\") pod \"aws-ebs-csi-driver-node-h2z7t\" (UID: \"e1c2402d-b45a-49ed-8f26-5cb19c6af258\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.043283 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.043260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6pmd\" (UniqueName: \"kubernetes.io/projected/6ebd1cb5-3def-4bae-9218-ceffe14067df-kube-api-access-z6pmd\") pod \"tuned-j4w9x\" (UID: \"6ebd1cb5-3def-4bae-9218-ceffe14067df\") " pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.043382 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.043329 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbnq4\" (UniqueName: \"kubernetes.io/projected/8e74d041-c34c-4f23-8952-a747e8c312a2-kube-api-access-kbnq4\") pod \"ovnkube-node-jcjq6\" (UID: \"8e74d041-c34c-4f23-8952-a747e8c312a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.045596 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.045572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98m5j\" (UniqueName: \"kubernetes.io/projected/1a335436-5134-4780-95ff-b3d182fb2969-kube-api-access-98m5j\") pod \"node-ca-6rsl6\" (UID: \"1a335436-5134-4780-95ff-b3d182fb2969\") " pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.045794 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.045766 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hm8h\" (UniqueName: \"kubernetes.io/projected/ddb47c98-02dd-4f7d-8830-d9f56f73f9e3-kube-api-access-5hm8h\") pod \"node-resolver-72cjz\" (UID: \"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3\") " pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.046039 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.046012 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gfn5\" (UniqueName: \"kubernetes.io/projected/04283751-84b7-4886-8a2e-e7e3085d2377-kube-api-access-2gfn5\") pod \"multus-additional-cni-plugins-ncx2q\" (UID: \"04283751-84b7-4886-8a2e-e7e3085d2377\") " pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.108309 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.108264 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1a81f81de33839e1fd7043ad57b780.slice/crio-f5d0df23ded7dbb6089c169e74801e00d805eaa1f6d0091c4fcae422e8f28353 WatchSource:0}: Error finding container f5d0df23ded7dbb6089c169e74801e00d805eaa1f6d0091c4fcae422e8f28353: Status 404 returned error can't find the container with id f5d0df23ded7dbb6089c169e74801e00d805eaa1f6d0091c4fcae422e8f28353 Jul 16 17:33:39.112055 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.112039 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:33:39.123808 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.123790 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:33:39.136443 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136419 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-hostroot\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136524 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:39.136524 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136472 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-hostroot\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136524 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136481 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-k8s-cni-cncf-io\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136524 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-netns\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136524 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62952831-854a-48e2-886c-3cdf2492c97a-kubelet-config\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.136558 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136567 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62952831-854a-48e2-886c-3cdf2492c97a-kubelet-config\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-k8s-cni-cncf-io\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-netns\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-cni-multus\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.136613 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:33:39.636595413 +0000 UTC m=+2.122888416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-cni-multus\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-daemon-config\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/380ce908-8115-4e31-9216-4ffded74c95e-iptables-alerter-script\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.136690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-cni-bin\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136714 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2dd25ad-b133-41cd-af46-3c7b865ea400-agent-certs\") pod \"konnectivity-agent-2mbhd\" (UID: \"f2dd25ad-b133-41cd-af46-3c7b865ea400\") " pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-kubelet\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.136763 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136768 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-kubelet\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136770 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-var-lib-cni-bin\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.136800 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:33:39.636790481 +0000 UTC m=+2.123083464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/380ce908-8115-4e31-9216-4ffded74c95e-host-slash\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-cni-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136894 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/380ce908-8115-4e31-9216-4ffded74c95e-host-slash\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-cnibin\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-conf-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136964 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-cni-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.136982 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ghw2\" (UniqueName: \"kubernetes.io/projected/380ce908-8115-4e31-9216-4ffded74c95e-kube-api-access-4ghw2\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137007 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-cnibin\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137061 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-conf-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x89dz\" (UniqueName: \"kubernetes.io/projected/84b3a525-e62d-44eb-b93f-2c8d1873420c-kube-api-access-x89dz\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16e7f2ae-f5aa-4116-a508-eaf0787a813a-cni-binary-copy\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137250 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-multus-certs\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137155 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62952831-854a-48e2-886c-3cdf2492c97a-dbus\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137178 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-system-cni-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-etc-kubernetes\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137245 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-daemon-config\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6j29\" (UniqueName: \"kubernetes.io/projected/16e7f2ae-f5aa-4116-a508-eaf0787a813a-kube-api-access-t6j29\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137277 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62952831-854a-48e2-886c-3cdf2492c97a-dbus\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137293 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-os-release\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137313 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-etc-kubernetes\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137317 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-socket-dir-parent\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137323 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-system-cni-dir\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2dd25ad-b133-41cd-af46-3c7b865ea400-konnectivity-ca\") pod \"konnectivity-agent-2mbhd\" (UID: \"f2dd25ad-b133-41cd-af46-3c7b865ea400\") " pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137363 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-multus-socket-dir-parent\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137319 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-host-run-multus-certs\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137387 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/16e7f2ae-f5aa-4116-a508-eaf0787a813a-os-release\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137648 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/380ce908-8115-4e31-9216-4ffded74c95e-iptables-alerter-script\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137756 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/16e7f2ae-f5aa-4116-a508-eaf0787a813a-cni-binary-copy\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.137876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.137860 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f2dd25ad-b133-41cd-af46-3c7b865ea400-konnectivity-ca\") pod \"konnectivity-agent-2mbhd\" (UID: \"f2dd25ad-b133-41cd-af46-3c7b865ea400\") " pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.139122 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.139106 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f2dd25ad-b133-41cd-af46-3c7b865ea400-agent-certs\") pod \"konnectivity-agent-2mbhd\" (UID: \"f2dd25ad-b133-41cd-af46-3c7b865ea400\") " pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.153327 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.153303 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6j29\" (UniqueName: \"kubernetes.io/projected/16e7f2ae-f5aa-4116-a508-eaf0787a813a-kube-api-access-t6j29\") pod \"multus-h8kcl\" (UID: \"16e7f2ae-f5aa-4116-a508-eaf0787a813a\") " pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.153797 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.153779 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ghw2\" (UniqueName: \"kubernetes.io/projected/380ce908-8115-4e31-9216-4ffded74c95e-kube-api-access-4ghw2\") pod \"iptables-alerter-tcbhq\" (UID: \"380ce908-8115-4e31-9216-4ffded74c95e\") " pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.154416 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.154391 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x89dz\" (UniqueName: \"kubernetes.io/projected/84b3a525-e62d-44eb-b93f-2c8d1873420c-kube-api-access-x89dz\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:39.154975 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.154957 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d13bff1d03965b595bba576b1d6f637.slice/crio-892faedb8f2fd407eaa0255d3594b1d59017cc10c7642d535192ee59e316ba1e WatchSource:0}: Error finding container 892faedb8f2fd407eaa0255d3594b1d59017cc10c7642d535192ee59e316ba1e: Status 404 returned error can't find the container with id 892faedb8f2fd407eaa0255d3594b1d59017cc10c7642d535192ee59e316ba1e Jul 16 17:33:39.238259 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.238188 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" Jul 16 17:33:39.245073 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.245054 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-72cjz" Jul 16 17:33:39.246941 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.246922 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebd1cb5_3def_4bae_9218_ceffe14067df.slice/crio-25c387cad193ee8e7e36493858421a15f6098203c784432c6e2b2b903d1a68e5 WatchSource:0}: Error finding container 25c387cad193ee8e7e36493858421a15f6098203c784432c6e2b2b903d1a68e5: Status 404 returned error can't find the container with id 25c387cad193ee8e7e36493858421a15f6098203c784432c6e2b2b903d1a68e5 Jul 16 17:33:39.253057 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.253017 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb47c98_02dd_4f7d_8830_d9f56f73f9e3.slice/crio-3eb4ad73cbcd41af78009440bd37fd2a8bd8d0085446f8d070c528f4a8b48e9c WatchSource:0}: Error finding container 3eb4ad73cbcd41af78009440bd37fd2a8bd8d0085446f8d070c528f4a8b48e9c: Status 404 returned error can't find the container with id 3eb4ad73cbcd41af78009440bd37fd2a8bd8d0085446f8d070c528f4a8b48e9c Jul 16 17:33:39.265713 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.265696 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rsl6" Jul 16 17:33:39.267085 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.267070 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:33:39.275375 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.275356 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a335436_5134_4780_95ff_b3d182fb2969.slice/crio-6d64720661b9fbf9a45089563006492343cf1e7cb98ca3839019cb4b5af4f95a WatchSource:0}: Error finding container 6d64720661b9fbf9a45089563006492343cf1e7cb98ca3839019cb4b5af4f95a: Status 404 returned error can't find the container with id 6d64720661b9fbf9a45089563006492343cf1e7cb98ca3839019cb4b5af4f95a Jul 16 17:33:39.277773 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.277755 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e74d041_c34c_4f23_8952_a747e8c312a2.slice/crio-46658f5700b39c16fce44435279c3e4cfe467d2b3c69e891b9944e04d685c4ab WatchSource:0}: Error finding container 46658f5700b39c16fce44435279c3e4cfe467d2b3c69e891b9944e04d685c4ab: Status 404 returned error can't find the container with id 46658f5700b39c16fce44435279c3e4cfe467d2b3c69e891b9944e04d685c4ab Jul 16 17:33:39.280657 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.280643 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" Jul 16 17:33:39.289335 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.289315 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c2402d_b45a_49ed_8f26_5cb19c6af258.slice/crio-eeb19a4cb77cb7deeb44697d39592d5a5c5682db9eeff1d868919f3a1b8dff12 WatchSource:0}: Error finding container eeb19a4cb77cb7deeb44697d39592d5a5c5682db9eeff1d868919f3a1b8dff12: Status 404 returned error can't find the container with id eeb19a4cb77cb7deeb44697d39592d5a5c5682db9eeff1d868919f3a1b8dff12 Jul 16 17:33:39.299093 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.299078 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" Jul 16 17:33:39.303965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.303947 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h8kcl" Jul 16 17:33:39.307064 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.307046 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04283751_84b7_4886_8a2e_e7e3085d2377.slice/crio-ecc304bf9f05ada627f50b6a263982f7c682a7df23b808d1d70e6fb54fb87d0c WatchSource:0}: Error finding container ecc304bf9f05ada627f50b6a263982f7c682a7df23b808d1d70e6fb54fb87d0c: Status 404 returned error can't find the container with id ecc304bf9f05ada627f50b6a263982f7c682a7df23b808d1d70e6fb54fb87d0c Jul 16 17:33:39.311799 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.311769 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e7f2ae_f5aa_4116_a508_eaf0787a813a.slice/crio-2c4aa13280b74fbef5071b9a10c5c14103cde8f569a73bf308f348bde4d04ebc WatchSource:0}: Error finding container 2c4aa13280b74fbef5071b9a10c5c14103cde8f569a73bf308f348bde4d04ebc: Status 404 returned error can't find the container with id 2c4aa13280b74fbef5071b9a10c5c14103cde8f569a73bf308f348bde4d04ebc Jul 16 17:33:39.329302 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.329285 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tcbhq" Jul 16 17:33:39.338240 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.338201 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380ce908_8115_4e31_9216_4ffded74c95e.slice/crio-8e3961ad0f0ac4e8f783170cdf07122edae2c1f6ddac3c1e06e93856d6266186 WatchSource:0}: Error finding container 8e3961ad0f0ac4e8f783170cdf07122edae2c1f6ddac3c1e06e93856d6266186: Status 404 returned error can't find the container with id 8e3961ad0f0ac4e8f783170cdf07122edae2c1f6ddac3c1e06e93856d6266186 Jul 16 17:33:39.339512 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.339491 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:33:39.347886 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:33:39.347867 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2dd25ad_b133_41cd_af46_3c7b865ea400.slice/crio-0d0679db5347d2c91776ef7bee31de08a8dc6f57b599fd4ee42cc4998d5f10e8 WatchSource:0}: Error finding container 0d0679db5347d2c91776ef7bee31de08a8dc6f57b599fd4ee42cc4998d5f10e8: Status 404 returned error can't find the container with id 0d0679db5347d2c91776ef7bee31de08a8dc6f57b599fd4ee42cc4998d5f10e8 Jul 16 17:33:39.356201 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.356183 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:33:39.541939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.541862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:39.542082 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.542011 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:33:39.542082 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.542041 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:33:39.542082 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.542049 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:39.542184 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.542100 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:33:40.542083746 +0000 UTC m=+3.028376747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:39.642565 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.642539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:39.642723 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.642574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:39.642723 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.642681 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:39.642723 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.642686 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:39.642829 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.642729 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:33:40.64271638 +0000 UTC m=+3.129009363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:39.642829 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:39.642742 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:33:40.642735551 +0000 UTC m=+3.129028534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:39.976293 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.976182 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 17:28:39 +0000 UTC" deadline="2028-02-14 18:41:56.255825709 +0000 UTC" Jul 16 17:33:39.976293 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:39.976248 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13873h8m16.279581939s" Jul 16 17:33:40.071514 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.071484 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:40.071680 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.071604 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:40.073850 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.073788 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2mbhd" event={"ID":"f2dd25ad-b133-41cd-af46-3c7b865ea400","Type":"ContainerStarted","Data":"0d0679db5347d2c91776ef7bee31de08a8dc6f57b599fd4ee42cc4998d5f10e8"} Jul 16 17:33:40.078427 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.078399 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tcbhq" event={"ID":"380ce908-8115-4e31-9216-4ffded74c95e","Type":"ContainerStarted","Data":"8e3961ad0f0ac4e8f783170cdf07122edae2c1f6ddac3c1e06e93856d6266186"} Jul 16 17:33:40.084545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.084522 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h8kcl" event={"ID":"16e7f2ae-f5aa-4116-a508-eaf0787a813a","Type":"ContainerStarted","Data":"2c4aa13280b74fbef5071b9a10c5c14103cde8f569a73bf308f348bde4d04ebc"} Jul 16 17:33:40.091282 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.091256 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerStarted","Data":"ecc304bf9f05ada627f50b6a263982f7c682a7df23b808d1d70e6fb54fb87d0c"} Jul 16 17:33:40.098877 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.098851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" event={"ID":"e1c2402d-b45a-49ed-8f26-5cb19c6af258","Type":"ContainerStarted","Data":"eeb19a4cb77cb7deeb44697d39592d5a5c5682db9eeff1d868919f3a1b8dff12"} Jul 16 17:33:40.100823 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.100797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rsl6" event={"ID":"1a335436-5134-4780-95ff-b3d182fb2969","Type":"ContainerStarted","Data":"6d64720661b9fbf9a45089563006492343cf1e7cb98ca3839019cb4b5af4f95a"} Jul 16 17:33:40.104043 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.104005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" event={"ID":"cd1a81f81de33839e1fd7043ad57b780","Type":"ContainerStarted","Data":"f5d0df23ded7dbb6089c169e74801e00d805eaa1f6d0091c4fcae422e8f28353"} Jul 16 17:33:40.106577 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.106554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"46658f5700b39c16fce44435279c3e4cfe467d2b3c69e891b9944e04d685c4ab"} Jul 16 17:33:40.110871 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.110848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-72cjz" event={"ID":"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3","Type":"ContainerStarted","Data":"3eb4ad73cbcd41af78009440bd37fd2a8bd8d0085446f8d070c528f4a8b48e9c"} Jul 16 17:33:40.115718 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.115686 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" event={"ID":"6ebd1cb5-3def-4bae-9218-ceffe14067df","Type":"ContainerStarted","Data":"25c387cad193ee8e7e36493858421a15f6098203c784432c6e2b2b903d1a68e5"} Jul 16 17:33:40.116997 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.116975 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" event={"ID":"5d13bff1d03965b595bba576b1d6f637","Type":"ContainerStarted","Data":"892faedb8f2fd407eaa0255d3594b1d59017cc10c7642d535192ee59e316ba1e"} Jul 16 17:33:40.418423 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.418337 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:33:40.550494 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.550461 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:40.550680 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.550604 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:33:40.550680 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.550621 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:33:40.550680 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.550630 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:40.550680 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.550677 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:33:42.550662203 +0000 UTC m=+5.036955187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:40.652698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.651801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:40.652698 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.651889 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:40.652698 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.652042 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:40.652698 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.652127 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:33:42.652106928 +0000 UTC m=+5.138399935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:40.652698 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.652601 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:40.652698 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:40.652660 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:33:42.652643216 +0000 UTC m=+5.138936203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:40.976870 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.976799 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 17:28:39 +0000 UTC" deadline="2028-03-01 08:53:58.890987888 +0000 UTC" Jul 16 17:33:40.976870 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:40.976839 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14247h20m17.914153435s" Jul 16 17:33:41.059418 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:41.059385 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:41.059604 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:41.059521 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:41.059979 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:41.059954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:41.060095 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:41.060072 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:42.059218 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:42.059183 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:42.059668 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.059342 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:42.571879 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:42.571836 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:42.572133 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.572093 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:33:42.572133 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.572118 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:33:42.572261 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.572159 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:42.572261 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.572235 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:33:46.572207638 +0000 UTC m=+9.058500640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:42.672775 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:42.672689 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:42.672775 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:42.672752 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:42.673012 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.672894 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:42.673012 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.672955 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:33:46.672936798 +0000 UTC m=+9.159229788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:42.673422 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.673402 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:42.673489 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:42.673463 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:33:46.673447366 +0000 UTC m=+9.159740356 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:43.058885 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:43.058801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:43.059091 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:43.059041 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:43.059553 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:43.059529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:43.059925 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:43.059636 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:44.063633 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:44.063598 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:44.064124 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:44.063731 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:45.058911 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:45.058869 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:45.059149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:45.058918 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:45.059149 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:45.059025 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:45.059278 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:45.059143 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:46.062230 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:46.061788 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:46.062230 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.061885 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:46.603378 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:46.603320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:46.603588 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.603571 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:33:46.603654 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.603597 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:33:46.603654 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.603610 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:46.603781 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.603684 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:33:54.603657276 +0000 UTC m=+17.089950263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:46.703832 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:46.703789 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:46.704016 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:46.703849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:46.704016 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.703985 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:46.704016 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.703988 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:46.704207 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.704052 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:33:54.704032359 +0000 UTC m=+17.190325345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:46.704207 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:46.704082 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:33:54.704062512 +0000 UTC m=+17.190355510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:47.059752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:47.059208 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:47.059752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:47.059233 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:47.059752 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:47.059356 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:47.059752 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:47.059484 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:48.060796 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:48.060286 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:48.060796 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:48.060399 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:49.058552 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:49.058470 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:49.058703 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:49.058652 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:49.059077 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:49.059059 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:49.059182 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:49.059161 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:50.058608 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:50.058573 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:50.059060 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:50.058707 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:51.058800 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:51.058770 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:51.059120 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:51.058884 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:51.059120 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:51.058936 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:51.059120 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:51.059035 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:52.058956 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:52.058706 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:52.059351 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:52.059018 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:53.058907 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:53.058874 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:53.059096 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:53.058877 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:53.059096 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:53.058997 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:53.059437 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:53.059088 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:54.058673 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:54.058489 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:54.058673 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.058615 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:54.664777 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:54.664734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:54.665256 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.664883 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:33:54.665256 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.664903 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:33:54.665256 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.664918 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:54.665256 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.664971 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:34:10.664957987 +0000 UTC m=+33.151250970 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:33:54.765916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:54.765870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:54.765916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:54.765928 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:54.766232 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.766057 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:54.766232 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.766057 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:54.766232 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.766125 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:34:10.766104024 +0000 UTC m=+33.252397008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:33:54.766232 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:54.766139 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:34:10.766132087 +0000 UTC m=+33.252425070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:33:55.058857 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:55.058777 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:55.059043 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:55.058797 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:55.059043 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:55.058905 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:55.059043 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:55.059007 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:56.059041 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:56.058991 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:56.059545 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:56.059139 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:57.059345 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:57.059301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:57.059345 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:57.059321 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:57.059797 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:57.059435 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:57.059797 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:57.059582 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:33:58.059071 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:58.059020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:33:58.059261 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:58.059170 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:33:59.058867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:59.058835 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:33:59.059295 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:59.058951 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:33:59.059295 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:33:59.059013 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:33:59.059295 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:33:59.059148 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:00.058792 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:00.058725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:00.058918 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:00.058851 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:01.059666 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.059397 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:01.060341 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.059400 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:01.060341 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:01.059740 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:01.060341 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:01.059846 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:01.173734 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.173696 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2mbhd" event={"ID":"f2dd25ad-b133-41cd-af46-3c7b865ea400","Type":"ContainerStarted","Data":"57c3b1763801b9f026508b4ed4f266893eee17d68019b1272d65b33a5590c8dc"} Jul 16 17:34:01.175156 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.175133 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:34:01.175313 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.175288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h8kcl" event={"ID":"16e7f2ae-f5aa-4116-a508-eaf0787a813a","Type":"ContainerStarted","Data":"7fd4366b1855d99cbbd9fde223adbcc52c4ce35dd04ff266d1e4bbd160c3830e"} Jul 16 17:34:01.176011 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.175989 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:34:01.177110 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.177091 2579 generic.go:356] "Generic (PLEG): container finished" podID="04283751-84b7-4886-8a2e-e7e3085d2377" containerID="af1faa3c3e5fea4591c0f5e89df5ddec885dd8fc6c45d2dc249613006c506c32" exitCode=0 Jul 16 17:34:01.177194 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.177165 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerDied","Data":"af1faa3c3e5fea4591c0f5e89df5ddec885dd8fc6c45d2dc249613006c506c32"} Jul 16 17:34:01.178837 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.178812 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" event={"ID":"e1c2402d-b45a-49ed-8f26-5cb19c6af258","Type":"ContainerStarted","Data":"049d1c07f6cd1929bd8ec9fcfe8f60b4222ec271b188446395797fb5dca709e1"} Jul 16 17:34:01.180360 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.180315 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rsl6" event={"ID":"1a335436-5134-4780-95ff-b3d182fb2969","Type":"ContainerStarted","Data":"1276cdcfb8bd7f47f371787a78044a3ce9e6251a7955d17dc50a57554757d208"} Jul 16 17:34:01.181720 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.181697 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" event={"ID":"cd1a81f81de33839e1fd7043ad57b780","Type":"ContainerStarted","Data":"c76df55537925fedac6fb94fcb2f3de794e56239fcd3be305816f6a7d32e1a3a"} Jul 16 17:34:01.184427 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.184411 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:34:01.184734 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.184710 2579 generic.go:356] "Generic (PLEG): container finished" podID="8e74d041-c34c-4f23-8952-a747e8c312a2" containerID="6fcf6d2d366b90daf1187115b76176e25765403f3d1233f3f5726533e06c7d8d" exitCode=1 Jul 16 17:34:01.184986 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.184792 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"82253279528e437709ec645e954fd791305c3696f9f9f16413553299bb0f7126"} Jul 16 17:34:01.184986 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.184870 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"39cc874e108306c0bf420ae12c15f5fd7779af79f5702761374cfb545dad4448"} Jul 16 17:34:01.184986 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.184921 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"aade62c2dda5a29704dcaeeda6bd73ab939035fb3691aee4af89a0fd7c88f9a7"} Jul 16 17:34:01.184986 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.184969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"13e900d92695184ec5d7b62049aa7cbfd98497ef01cd147cadab3c227b5c7064"} Jul 16 17:34:01.185221 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.185005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerDied","Data":"6fcf6d2d366b90daf1187115b76176e25765403f3d1233f3f5726533e06c7d8d"} Jul 16 17:34:01.185221 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.185075 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"4cd26de3bb8ca8da580d6ea0e7e10975593ab29927a3c5c7de6125272ae29529"} Jul 16 17:34:01.186214 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.186181 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-72cjz" event={"ID":"ddb47c98-02dd-4f7d-8830-d9f56f73f9e3","Type":"ContainerStarted","Data":"9ba399430cdbdc1f6cdc120cfa52ccf7bde8705b5f27ac30fd3f64850d1beaa2"} Jul 16 17:34:01.187939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.187917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" event={"ID":"6ebd1cb5-3def-4bae-9218-ceffe14067df","Type":"ContainerStarted","Data":"3ad549b09f7057f8995ea2a51f63ca221c0a22e565e398edef7bd009ecbbd74d"} Jul 16 17:34:01.189461 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.189441 2579 generic.go:356] "Generic (PLEG): container finished" podID="5d13bff1d03965b595bba576b1d6f637" containerID="171c96d2ea8d2fade3502385af0dd4bd380d64ffdb1975ff4e46db4b045829cb" exitCode=0 Jul 16 17:34:01.189531 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.189470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" event={"ID":"5d13bff1d03965b595bba576b1d6f637","Type":"ContainerDied","Data":"171c96d2ea8d2fade3502385af0dd4bd380d64ffdb1975ff4e46db4b045829cb"} Jul 16 17:34:01.191874 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.191841 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2mbhd" podStartSLOduration=1.203315421 podStartE2EDuration="22.191831298s" totalImagesPullingTime="20.988515877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:01.191506077 +0000 UTC m=+23.677799078" watchObservedRunningTime="2026-07-16 17:34:01.191831298 +0000 UTC m=+23.678124303" Jul 16 17:34:01.209658 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.209617 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h8kcl" podStartSLOduration=2.061484614 podStartE2EDuration="23.209605289s" totalImagesPullingTime="21.148120675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:01.209262966 +0000 UTC m=+23.695555970" watchObservedRunningTime="2026-07-16 17:34:01.209605289 +0000 UTC m=+23.695898293" Jul 16 17:34:01.225071 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.225035 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-72cjz" podStartSLOduration=2.155566332 podStartE2EDuration="23.225008491s" totalImagesPullingTime="21.069442159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:01.224849575 +0000 UTC m=+23.711142580" watchObservedRunningTime="2026-07-16 17:34:01.225008491 +0000 UTC m=+23.711301495" Jul 16 17:34:01.284923 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.284877 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-151.ec2.internal" podStartSLOduration=22.284862177 podStartE2EDuration="22.284862177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:01.284641343 +0000 UTC m=+23.770934345" watchObservedRunningTime="2026-07-16 17:34:01.284862177 +0000 UTC m=+23.771155222" Jul 16 17:34:01.304758 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.304716 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j4w9x" podStartSLOduration=2.163022158 podStartE2EDuration="23.304702884s" totalImagesPullingTime="21.141680726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:01.304269509 +0000 UTC m=+23.790562513" watchObservedRunningTime="2026-07-16 17:34:01.304702884 +0000 UTC m=+23.790995939" Jul 16 17:34:01.323857 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:01.323814 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6rsl6" podStartSLOduration=2.26294753 podStartE2EDuration="23.323801239s" totalImagesPullingTime="21.060853709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:01.323634547 +0000 UTC m=+23.809927553" watchObservedRunningTime="2026-07-16 17:34:01.323801239 +0000 UTC m=+23.810094244" Jul 16 17:34:02.062466 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.062443 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:02.062929 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:02.062568 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:02.144114 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.144094 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 17:34:02.193112 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.193076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tcbhq" event={"ID":"380ce908-8115-4e31-9216-4ffded74c95e","Type":"ContainerStarted","Data":"af193f1e77338303d54127913dfd34209eeb00073920481fe0b19e7cdbfa5b5b"} Jul 16 17:34:02.198249 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.198216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" event={"ID":"e1c2402d-b45a-49ed-8f26-5cb19c6af258","Type":"ContainerStarted","Data":"09faa5e15722393d12c2408902275279990f0378301bff690fec5c29e0af5e7e"} Jul 16 17:34:02.199090 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.199066 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:34:02.199195 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.199142 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2mbhd" Jul 16 17:34:02.212381 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:02.212336 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tcbhq" podStartSLOduration=3.16119132 podStartE2EDuration="24.212318573s" totalImagesPullingTime="21.051127253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:02.21203892 +0000 UTC m=+24.698331920" watchObservedRunningTime="2026-07-16 17:34:02.212318573 +0000 UTC m=+24.698611577" Jul 16 17:34:03.004353 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.004272 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T17:34:02.144110729Z","UUID":"0160dc45-632d-47d1-9e8c-4f03f97dc1db","Handler":null,"Name":"","Endpoint":""} Jul 16 17:34:03.005814 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.005794 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 17:34:03.005948 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.005833 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 17:34:03.059099 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.059063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:03.059267 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:03.059202 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:03.059267 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.059257 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:03.059397 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:03.059366 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:03.205363 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.205333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" event={"ID":"5d13bff1d03965b595bba576b1d6f637","Type":"ContainerStarted","Data":"e3006c3191d210ab3c6ad3e045362a208b586d5e71a337827e59eb00ca41cc16"} Jul 16 17:34:03.231586 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:03.231538 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-151.ec2.internal" podStartSLOduration=24.231517707 podStartE2EDuration="24.231517707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:03.231000237 +0000 UTC m=+25.717293243" watchObservedRunningTime="2026-07-16 17:34:03.231517707 +0000 UTC m=+25.717810712" Jul 16 17:34:04.062818 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:04.062578 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:04.062973 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:04.062922 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:04.210859 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:04.210832 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:34:04.211339 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:04.211313 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"0128228dc6866b91165bc8b8fbe47808879dcf28fe226a3aaa30b8ccbb9f64ff"} Jul 16 17:34:05.058847 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:05.058811 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:05.059058 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:05.058853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:05.059058 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:05.058937 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:05.059058 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:05.059014 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:06.058802 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.058619 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:06.059636 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:06.058883 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:06.219300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.219274 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:34:06.219640 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.219619 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"e274781d4aae2c5243f2e10b9d49b6892d9507f5a8a7133b80a04b7c391684ef"} Jul 16 17:34:06.219890 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.219864 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:34:06.220051 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.220018 2579 scope.go:117] "RemoveContainer" containerID="6fcf6d2d366b90daf1187115b76176e25765403f3d1233f3f5726533e06c7d8d" Jul 16 17:34:06.221505 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.221481 2579 generic.go:356] "Generic (PLEG): container finished" podID="04283751-84b7-4886-8a2e-e7e3085d2377" containerID="aecdce2a5c89162e088d7a171c9f6e15e099d283d202760886e413aadc4566b6" exitCode=0 Jul 16 17:34:06.221599 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.221552 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerDied","Data":"aecdce2a5c89162e088d7a171c9f6e15e099d283d202760886e413aadc4566b6"} Jul 16 17:34:06.224039 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.223998 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" event={"ID":"e1c2402d-b45a-49ed-8f26-5cb19c6af258","Type":"ContainerStarted","Data":"19f3d6d3ef83772690a060333ffd89bbafdf87adfbe39bc08faefe2afae29727"} Jul 16 17:34:06.237210 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.237195 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:34:06.321622 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:06.321541 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2z7t" podStartSLOduration=2.933915636 podStartE2EDuration="28.321528556s" totalImagesPullingTime="25.38761292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:06.320771097 +0000 UTC m=+28.807064093" watchObservedRunningTime="2026-07-16 17:34:06.321528556 +0000 UTC m=+28.807821560" Jul 16 17:34:07.059497 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.059451 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:07.060004 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.059482 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:07.060004 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:07.059605 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:07.060004 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:07.059719 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:07.230598 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.229955 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:34:07.230598 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.230398 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" event={"ID":"8e74d041-c34c-4f23-8952-a747e8c312a2","Type":"ContainerStarted","Data":"f453c9a52532ae2dfabafe00e71a02fd94184720809982d2dc4ae12b238a1da7"} Jul 16 17:34:07.230817 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.230790 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:34:07.230922 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.230876 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:34:07.247150 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.247122 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:34:07.265533 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.265474 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" podStartSLOduration=8.132089713 podStartE2EDuration="29.265455191s" totalImagesPullingTime="21.133365478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:07.264332657 +0000 UTC m=+29.750625662" watchObservedRunningTime="2026-07-16 17:34:07.265455191 +0000 UTC m=+29.751748197" Jul 16 17:34:07.589278 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.589210 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nm4p2"] Jul 16 17:34:07.589483 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.589331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:07.589483 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:07.589432 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:07.591155 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.591128 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b87sw"] Jul 16 17:34:07.591261 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.591211 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:07.591324 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:07.591303 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:07.591739 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.591720 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4tq6n"] Jul 16 17:34:07.591819 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:07.591800 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:07.591909 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:07.591889 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:08.234289 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:08.234252 2579 generic.go:356] "Generic (PLEG): container finished" podID="04283751-84b7-4886-8a2e-e7e3085d2377" containerID="888538b7ba52aad0f5db724b5c73abeab00d76ae82489f9361f2ecf0b21e3449" exitCode=0 Jul 16 17:34:08.234706 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:08.234333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerDied","Data":"888538b7ba52aad0f5db724b5c73abeab00d76ae82489f9361f2ecf0b21e3449"} Jul 16 17:34:09.058888 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:09.058851 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:09.058888 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:09.058887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:09.059139 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:09.058967 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:09.059139 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:09.059099 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:10.058751 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:10.058585 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:10.059137 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.058851 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:10.241458 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:10.241428 2579 generic.go:356] "Generic (PLEG): container finished" podID="04283751-84b7-4886-8a2e-e7e3085d2377" containerID="91b6bd50bd28644d879678e5fe5d55aea055d9dc834a2f4aedd8ba6b71b79c00" exitCode=0 Jul 16 17:34:10.241599 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:10.241477 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerDied","Data":"91b6bd50bd28644d879678e5fe5d55aea055d9dc834a2f4aedd8ba6b71b79c00"} Jul 16 17:34:10.686356 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:10.686322 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:10.686512 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.686442 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:34:10.686512 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.686455 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:34:10.686512 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.686463 2579 projected.go:196] Error preparing data for projected volume kube-api-access-dqq4x for pod openshift-network-diagnostics/network-check-target-b87sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:34:10.686512 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.686510 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x podName:d3fa8548-aca3-4884-9081-b61951f36844 nodeName:}" failed. No retries permitted until 2026-07-16 17:34:42.686497673 +0000 UTC m=+65.172790656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqq4x" (UniqueName: "kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x") pod "network-check-target-b87sw" (UID: "d3fa8548-aca3-4884-9081-b61951f36844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:34:10.787324 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:10.787290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:10.787324 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:10.787327 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:10.787558 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.787430 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:34:10.787558 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.787437 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:34:10.787558 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.787487 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret podName:62952831-854a-48e2-886c-3cdf2492c97a nodeName:}" failed. No retries permitted until 2026-07-16 17:34:42.787471268 +0000 UTC m=+65.273764271 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret") pod "global-pull-secret-syncer-4tq6n" (UID: "62952831-854a-48e2-886c-3cdf2492c97a") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:34:10.787558 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:10.787511 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:34:42.787504714 +0000 UTC m=+65.273797697 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:34:11.058575 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:11.058517 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:11.058864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:11.058522 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:11.058864 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:11.058632 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:11.058864 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:11.058746 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:12.059488 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:12.059458 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:12.059988 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:12.059596 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nm4p2" podUID="84b3a525-e62d-44eb-b93f-2c8d1873420c" Jul 16 17:34:13.059084 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.059050 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:13.059239 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.059050 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:13.059239 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.059194 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b87sw" podUID="d3fa8548-aca3-4884-9081-b61951f36844" Jul 16 17:34:13.059333 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.059260 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4tq6n" podUID="62952831-854a-48e2-886c-3cdf2492c97a" Jul 16 17:34:13.370751 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.370719 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-151.ec2.internal" event="NodeReady" Jul 16 17:34:13.371199 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.370858 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 17:34:13.413995 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.413963 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f5f9c65dc-lk5lx"] Jul 16 17:34:13.416456 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.416434 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.422269 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.422249 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 17:34:13.422400 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.422300 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gcp44\"" Jul 16 17:34:13.425703 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.425683 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 17:34:13.425986 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.425966 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 17:34:13.449120 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.449091 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 17:34:13.462648 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.462619 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f5f9c65dc-lk5lx"] Jul 16 17:34:13.465748 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.465723 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5dj4t"] Jul 16 17:34:13.467709 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.467684 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.467836 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.467819 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m24mr"] Jul 16 17:34:13.469725 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.469707 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.472735 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.472707 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 17:34:13.472735 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.472726 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n568h\"" Jul 16 17:34:13.473785 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.473769 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ntp5m\"" Jul 16 17:34:13.474147 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.474126 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 17:34:13.474224 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.474202 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 17:34:13.475303 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.475287 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 17:34:13.475766 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.475747 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 17:34:13.492585 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.492565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dj4t"] Jul 16 17:34:13.507522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507500 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m24mr"] Jul 16 17:34:13.507762 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4ea9957-783c-4c4a-a665-7027a9a8caae-ca-trust-extracted\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.507826 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-trusted-ca\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.507826 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507799 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-image-registry-private-configuration\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.507918 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq6d6\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-kube-api-access-vq6d6\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.507918 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-installation-pull-secrets\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.508008 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.507948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-certificates\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.508095 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.508006 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.508095 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.508045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-bound-sa-token\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.608848 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.608809 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.608848 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.608858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3615bce5-3d4d-41a5-b5dc-4e460c50ba80-cert\") pod \"ingress-canary-5dj4t\" (UID: \"3615bce5-3d4d-41a5-b5dc-4e460c50ba80\") " pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.608888 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-certificates\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.608954 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e4221d61-f6c0-43bd-b6ac-38739ab0097e-config-volume\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609006 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-bound-sa-token\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf95v\" (UniqueName: \"kubernetes.io/projected/e4221d61-f6c0-43bd-b6ac-38739ab0097e-kube-api-access-jf95v\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609098 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4ea9957-783c-4c4a-a665-7027a9a8caae-ca-trust-extracted\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.609117 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609112 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-trusted-ca\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.609431 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4221d61-f6c0-43bd-b6ac-38739ab0097e-tmp-dir\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609486 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-certificates\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609523 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4ea9957-783c-4c4a-a665-7027a9a8caae-ca-trust-extracted\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.609611 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.609628 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.609681 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:34:14.109667266 +0000 UTC m=+36.595960249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609803 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwgph\" (UniqueName: \"kubernetes.io/projected/3615bce5-3d4d-41a5-b5dc-4e460c50ba80-kube-api-access-cwgph\") pod \"ingress-canary-5dj4t\" (UID: \"3615bce5-3d4d-41a5-b5dc-4e460c50ba80\") " pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-image-registry-private-configuration\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq6d6\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-kube-api-access-vq6d6\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.609901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-installation-pull-secrets\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.610500 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.610271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-trusted-ca\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.614002 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.613981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-installation-pull-secrets\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.614142 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.613988 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-image-registry-private-configuration\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.618587 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.618557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq6d6\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-kube-api-access-vq6d6\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.618760 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.618737 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-bound-sa-token\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:13.711038 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.710933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3615bce5-3d4d-41a5-b5dc-4e460c50ba80-cert\") pod \"ingress-canary-5dj4t\" (UID: \"3615bce5-3d4d-41a5-b5dc-4e460c50ba80\") " pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.711038 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.710990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e4221d61-f6c0-43bd-b6ac-38739ab0097e-config-volume\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.711256 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.711063 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf95v\" (UniqueName: \"kubernetes.io/projected/e4221d61-f6c0-43bd-b6ac-38739ab0097e-kube-api-access-jf95v\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.711256 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.711097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4221d61-f6c0-43bd-b6ac-38739ab0097e-tmp-dir\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.711256 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.711124 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwgph\" (UniqueName: \"kubernetes.io/projected/3615bce5-3d4d-41a5-b5dc-4e460c50ba80-kube-api-access-cwgph\") pod \"ingress-canary-5dj4t\" (UID: \"3615bce5-3d4d-41a5-b5dc-4e460c50ba80\") " pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.711256 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.711201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.711445 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.711308 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:13.711445 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:13.711374 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:34:14.211354983 +0000 UTC m=+36.697647971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:13.711700 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.711647 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e4221d61-f6c0-43bd-b6ac-38739ab0097e-config-volume\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.711700 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.711671 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4221d61-f6c0-43bd-b6ac-38739ab0097e-tmp-dir\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.713904 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.713881 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3615bce5-3d4d-41a5-b5dc-4e460c50ba80-cert\") pod \"ingress-canary-5dj4t\" (UID: \"3615bce5-3d4d-41a5-b5dc-4e460c50ba80\") " pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.720875 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.720803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf95v\" (UniqueName: \"kubernetes.io/projected/e4221d61-f6c0-43bd-b6ac-38739ab0097e-kube-api-access-jf95v\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:13.720875 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.720806 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwgph\" (UniqueName: \"kubernetes.io/projected/3615bce5-3d4d-41a5-b5dc-4e460c50ba80-kube-api-access-cwgph\") pod \"ingress-canary-5dj4t\" (UID: \"3615bce5-3d4d-41a5-b5dc-4e460c50ba80\") " pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:13.778822 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:13.778789 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dj4t" Jul 16 17:34:14.062597 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:14.062527 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:14.065563 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:14.065351 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 17:34:14.065563 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:14.065381 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9hcv\"" Jul 16 17:34:14.115217 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:14.115177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:14.115383 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:14.115336 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:14.115383 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:14.115350 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:14.115488 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:14.115405 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:34:15.115389636 +0000 UTC m=+37.601682826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:14.216575 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:14.216533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:14.216721 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:14.216687 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:14.216776 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:14.216759 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:34:15.216738944 +0000 UTC m=+37.703031943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:14.510589 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:14.510387 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dj4t"] Jul 16 17:34:14.515177 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:34:14.515155 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3615bce5_3d4d_41a5_b5dc_4e460c50ba80.slice/crio-a16aa06a3514af7640565a70cb12494ff5297f20bde6a743e89308b261042b03 WatchSource:0}: Error finding container a16aa06a3514af7640565a70cb12494ff5297f20bde6a743e89308b261042b03: Status 404 returned error can't find the container with id a16aa06a3514af7640565a70cb12494ff5297f20bde6a743e89308b261042b03 Jul 16 17:34:15.058554 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.058514 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:15.058739 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.058525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:15.061328 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.061289 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 17:34:15.062302 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.062286 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 17:34:15.062431 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.062306 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s9dx5\"" Jul 16 17:34:15.062431 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.062326 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 17:34:15.126146 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.126124 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:15.126272 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:15.126224 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:15.126272 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:15.126235 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:15.126336 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:15.126281 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:34:17.126268198 +0000 UTC m=+39.612561182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:15.227108 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.227080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:15.227233 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:15.227196 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:15.227288 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:15.227279 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:34:17.227264898 +0000 UTC m=+39.713557882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:15.254987 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.254949 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dj4t" event={"ID":"3615bce5-3d4d-41a5-b5dc-4e460c50ba80","Type":"ContainerStarted","Data":"a16aa06a3514af7640565a70cb12494ff5297f20bde6a743e89308b261042b03"} Jul 16 17:34:15.257514 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.257493 2579 generic.go:356] "Generic (PLEG): container finished" podID="04283751-84b7-4886-8a2e-e7e3085d2377" containerID="527ef8bc8cf9d5372140bd91949020b4391b7c2e7f8160d9cda57099f9cfdc92" exitCode=0 Jul 16 17:34:15.257615 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:15.257533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerDied","Data":"527ef8bc8cf9d5372140bd91949020b4391b7c2e7f8160d9cda57099f9cfdc92"} Jul 16 17:34:16.262174 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:16.262141 2579 generic.go:356] "Generic (PLEG): container finished" podID="04283751-84b7-4886-8a2e-e7e3085d2377" containerID="6d6ceb56610e4b3bf2dbf380e9cdeeac0d7048984930948f20b2af7a9af0c98f" exitCode=0 Jul 16 17:34:16.262549 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:16.262183 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerDied","Data":"6d6ceb56610e4b3bf2dbf380e9cdeeac0d7048984930948f20b2af7a9af0c98f"} Jul 16 17:34:16.699498 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:16.699397 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:17.143087 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.143040 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:17.143239 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:17.143188 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:17.143239 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:17.143208 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:17.143318 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:17.143262 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:34:21.143248044 +0000 UTC m=+43.629541027 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:17.244613 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.244376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:17.244759 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:17.244518 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:17.244759 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:17.244740 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:34:21.24471939 +0000 UTC m=+43.731012376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:17.269134 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.269096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" event={"ID":"04283751-84b7-4886-8a2e-e7e3085d2377","Type":"ContainerStarted","Data":"b00cef0547e31764e331c57a42f5ad4fce02fd89aab3d41e619884c35c357f12"} Jul 16 17:34:17.296574 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.296530 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ncx2q" podStartSLOduration=16.41664055 podStartE2EDuration="39.296514614s" totalImagesPullingTime="22.879874064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:17.296165894 +0000 UTC m=+39.782458899" watchObservedRunningTime="2026-07-16 17:34:17.296514614 +0000 UTC m=+39.782807597" Jul 16 17:34:17.449597 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.449517 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:17.531395 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.531360 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:17.601054 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:17.600980 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:19.004969 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:19.004935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:19.274995 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:19.274911 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dj4t" event={"ID":"3615bce5-3d4d-41a5-b5dc-4e460c50ba80","Type":"ContainerStarted","Data":"adbf4e7586c52835b36012c2cb641e055a92d8cdcd06adb84a97c4ea999f8005"} Jul 16 17:34:19.293915 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:19.293870 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5dj4t" podStartSLOduration=2.493780834 podStartE2EDuration="6.293856874s" totalImagesPullingTime="3.80007604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:19.292905553 +0000 UTC m=+41.779198561" watchObservedRunningTime="2026-07-16 17:34:19.293856874 +0000 UTC m=+41.780149879" Jul 16 17:34:20.827442 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:20.827402 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:21.172540 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:21.172445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:21.172713 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:21.172590 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:21.172713 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:21.172607 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:21.172713 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:21.172672 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:34:29.172653954 +0000 UTC m=+51.658946937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:21.273156 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:21.273118 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:21.273323 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:21.273282 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:21.273382 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:21.273362 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:34:29.273340739 +0000 UTC m=+51.759633749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:23.561728 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:23.561682 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:23.666902 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:23.666862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:27.957765 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:27.957727 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:28.044295 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:28.044263 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:28.107158 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:28.107123 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:29.231083 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:29.231040 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:29.231454 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:29.231135 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:29.231454 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:29.231147 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:29.231454 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:29.231204 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:34:45.231188492 +0000 UTC m=+67.717481477 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:29.331670 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:29.331640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:29.331833 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:29.331761 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:29.331833 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:29.331816 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:34:45.331803643 +0000 UTC m=+67.818096626 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:32.835571 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:32.835534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:32.935798 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:32.935737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:33.831309 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:33.831265 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:33.898828 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:33.898789 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:39.259065 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:39.259038 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jcjq6" Jul 16 17:34:39.421067 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:39.421006 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:42.726380 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.726331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:42.728668 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.728649 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 17:34:42.738845 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.738825 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 17:34:42.750790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.750759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqq4x\" (UniqueName: \"kubernetes.io/projected/d3fa8548-aca3-4884-9081-b61951f36844-kube-api-access-dqq4x\") pod \"network-check-target-b87sw\" (UID: \"d3fa8548-aca3-4884-9081-b61951f36844\") " pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:42.803405 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.803375 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:42.827476 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.827453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:34:42.827561 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.827489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:42.830247 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.830230 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 17:34:42.830341 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.830321 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 17:34:42.838267 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:42.838251 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 17:34:42.838318 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:42.838308 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs podName:84b3a525-e62d-44eb-b93f-2c8d1873420c nodeName:}" failed. No retries permitted until 2026-07-16 17:35:46.838293923 +0000 UTC m=+129.324586906 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs") pod "network-metrics-daemon-nm4p2" (UID: "84b3a525-e62d-44eb-b93f-2c8d1873420c") : secret "metrics-daemon-secret" not found Jul 16 17:34:42.840432 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.840409 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62952831-854a-48e2-886c-3cdf2492c97a-original-pull-secret\") pod \"global-pull-secret-syncer-4tq6n\" (UID: \"62952831-854a-48e2-886c-3cdf2492c97a\") " pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:42.968651 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.968621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4tq6n" Jul 16 17:34:42.974840 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.974820 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s9dx5\"" Jul 16 17:34:42.983017 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:42.982995 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:43.093824 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:43.093790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4tq6n"] Jul 16 17:34:43.097477 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:34:43.097449 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62952831_854a_48e2_886c_3cdf2492c97a.slice/crio-26d78334d83198342d52366668da5d78068f95be77ea6a1ad77a0f9a42057b0c WatchSource:0}: Error finding container 26d78334d83198342d52366668da5d78068f95be77ea6a1ad77a0f9a42057b0c: Status 404 returned error can't find the container with id 26d78334d83198342d52366668da5d78068f95be77ea6a1ad77a0f9a42057b0c Jul 16 17:34:43.109572 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:43.109551 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b87sw"] Jul 16 17:34:43.112312 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:34:43.112288 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fa8548_aca3_4884_9081_b61951f36844.slice/crio-0a054688e52169c4318fd7ae368ef764829819792dcb99ee1566776ae96eddc2 WatchSource:0}: Error finding container 0a054688e52169c4318fd7ae368ef764829819792dcb99ee1566776ae96eddc2: Status 404 returned error can't find the container with id 0a054688e52169c4318fd7ae368ef764829819792dcb99ee1566776ae96eddc2 Jul 16 17:34:43.330505 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:43.330416 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b87sw" event={"ID":"d3fa8548-aca3-4884-9081-b61951f36844","Type":"ContainerStarted","Data":"0a054688e52169c4318fd7ae368ef764829819792dcb99ee1566776ae96eddc2"} Jul 16 17:34:43.331515 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:43.331492 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4tq6n" event={"ID":"62952831-854a-48e2-886c-3cdf2492c97a","Type":"ContainerStarted","Data":"26d78334d83198342d52366668da5d78068f95be77ea6a1ad77a0f9a42057b0c"} Jul 16 17:34:45.245182 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:45.245050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:34:45.245700 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:45.245228 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:34:45.245700 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:45.245249 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:34:45.245700 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:45.245308 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:35:17.245292112 +0000 UTC m=+99.731585096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:34:45.345942 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:45.345913 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:34:45.346142 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:45.346040 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:34:45.346142 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:34:45.346107 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:35:17.346090205 +0000 UTC m=+99.832383188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:34:45.873491 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:45.873458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:45.935392 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:45.935345 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:49.349135 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:49.349098 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b87sw" event={"ID":"d3fa8548-aca3-4884-9081-b61951f36844","Type":"ContainerStarted","Data":"e954c4750db05e58b8826ae87c5a91b4dfe30d2458484ed82792b12106b7b2ca"} Jul 16 17:34:49.349613 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:49.349180 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:34:49.350638 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:49.350605 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4tq6n" event={"ID":"62952831-854a-48e2-886c-3cdf2492c97a","Type":"ContainerStarted","Data":"dc06abb936e55ccf76e87aa5ae612aec907a62da63b695d322d96011ec555fc3"} Jul 16 17:34:49.367774 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:49.367722 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b87sw" podStartSLOduration=66.229313586 podStartE2EDuration="1m11.367701911s" totalImagesPullingTime="5.138388325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:49.366707152 +0000 UTC m=+71.853000158" watchObservedRunningTime="2026-07-16 17:34:49.367701911 +0000 UTC m=+71.853994917" Jul 16 17:34:49.384348 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:49.384311 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4tq6n" podStartSLOduration=65.219835301 podStartE2EDuration="1m10.384298378s" totalImagesPullingTime="5.164463077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:49.383932846 +0000 UTC m=+71.870225852" watchObservedRunningTime="2026-07-16 17:34:49.384298378 +0000 UTC m=+71.870591383" Jul 16 17:34:50.082831 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:50.082801 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:50.855864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:50.855827 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:50.918206 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:50.918173 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:53.910541 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:53.910507 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:54.283094 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:34:54.283060 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:08.849890 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:08.849852 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:08.906438 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:08.906405 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:15.550508 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.550476 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr"] Jul 16 17:35:15.553006 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.552991 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:15.557595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.557562 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 17:35:15.558110 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.558089 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:35:15.558784 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.558553 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 17:35:15.558844 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.558818 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p4r6k\"" Jul 16 17:35:15.565149 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.565129 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr"] Jul 16 17:35:15.650167 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.650133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95md2\" (UniqueName: \"kubernetes.io/projected/3a732f00-b481-4210-8a1c-fba79fe5df59-kube-api-access-95md2\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:15.650333 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.650183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:15.650919 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.650901 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn"] Jul 16 17:35:15.652635 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.652622 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.655212 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.655190 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:35:15.655212 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.655204 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 17:35:15.655368 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.655192 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vgmxq\"" Jul 16 17:35:15.655701 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.655689 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 17:35:15.655933 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.655921 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 17:35:15.672444 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.672415 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn"] Jul 16 17:35:15.751178 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.751146 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.751331 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.751183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.751331 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.751201 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gktpj\" (UniqueName: \"kubernetes.io/projected/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-kube-api-access-gktpj\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.751331 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.751229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95md2\" (UniqueName: \"kubernetes.io/projected/3a732f00-b481-4210-8a1c-fba79fe5df59-kube-api-access-95md2\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:15.751331 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.751298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:15.751467 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:15.751426 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:35:15.751499 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:15.751490 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls podName:3a732f00-b481-4210-8a1c-fba79fe5df59 nodeName:}" failed. No retries permitted until 2026-07-16 17:35:16.251475807 +0000 UTC m=+98.737768812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls") pod "cluster-samples-operator-65db858894-svmmr" (UID: "3a732f00-b481-4210-8a1c-fba79fe5df59") : secret "samples-operator-tls" not found Jul 16 17:35:15.759851 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.759828 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95md2\" (UniqueName: \"kubernetes.io/projected/3a732f00-b481-4210-8a1c-fba79fe5df59-kube-api-access-95md2\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:15.821370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.821274 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:15.852354 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.852329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.852427 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.852364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.852427 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.852381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gktpj\" (UniqueName: \"kubernetes.io/projected/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-kube-api-access-gktpj\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.852964 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.852914 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.854588 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.854561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.861457 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.861438 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gktpj\" (UniqueName: \"kubernetes.io/projected/6c7131e0-7ce7-42a3-8707-b8af0419d0f7-kube-api-access-gktpj\") pod \"kube-storage-version-migrator-operator-799cb7f958-stvkn\" (UID: \"6c7131e0-7ce7-42a3-8707-b8af0419d0f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:15.889943 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.889917 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:15.961116 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:15.961089 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" Jul 16 17:35:16.080475 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:16.080410 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn"] Jul 16 17:35:16.083648 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:35:16.083624 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7131e0_7ce7_42a3_8707_b8af0419d0f7.slice/crio-8aa34cd5193c3dbe87cbc3599ed7473afc0dbbe4c15a08f81cfa76cf84ca3a13 WatchSource:0}: Error finding container 8aa34cd5193c3dbe87cbc3599ed7473afc0dbbe4c15a08f81cfa76cf84ca3a13: Status 404 returned error can't find the container with id 8aa34cd5193c3dbe87cbc3599ed7473afc0dbbe4c15a08f81cfa76cf84ca3a13 Jul 16 17:35:16.255176 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:16.255150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:16.255325 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:16.255286 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:35:16.255372 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:16.255342 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls podName:3a732f00-b481-4210-8a1c-fba79fe5df59 nodeName:}" failed. No retries permitted until 2026-07-16 17:35:17.255328641 +0000 UTC m=+99.741621624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls") pod "cluster-samples-operator-65db858894-svmmr" (UID: "3a732f00-b481-4210-8a1c-fba79fe5df59") : secret "samples-operator-tls" not found Jul 16 17:35:16.416353 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:16.416274 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" event={"ID":"6c7131e0-7ce7-42a3-8707-b8af0419d0f7","Type":"ContainerStarted","Data":"8aa34cd5193c3dbe87cbc3599ed7473afc0dbbe4c15a08f81cfa76cf84ca3a13"} Jul 16 17:35:17.264172 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:17.264137 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:17.264546 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:17.264207 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") pod \"image-registry-5f5f9c65dc-lk5lx\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:35:17.264546 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.264311 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:35:17.264546 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.264335 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 17:35:17.264546 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.264350 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5f9c65dc-lk5lx: secret "image-registry-tls" not found Jul 16 17:35:17.264546 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.264385 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls podName:3a732f00-b481-4210-8a1c-fba79fe5df59 nodeName:}" failed. No retries permitted until 2026-07-16 17:35:19.264367798 +0000 UTC m=+101.750660781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls") pod "cluster-samples-operator-65db858894-svmmr" (UID: "3a732f00-b481-4210-8a1c-fba79fe5df59") : secret "samples-operator-tls" not found Jul 16 17:35:17.264546 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.264412 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls podName:e4ea9957-783c-4c4a-a665-7027a9a8caae nodeName:}" failed. No retries permitted until 2026-07-16 17:36:21.264392959 +0000 UTC m=+163.750685982 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls") pod "image-registry-5f5f9c65dc-lk5lx" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae") : secret "image-registry-tls" not found Jul 16 17:35:17.365073 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:17.365018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:35:17.365245 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.365159 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:35:17.365245 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:17.365222 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls podName:e4221d61-f6c0-43bd-b6ac-38739ab0097e nodeName:}" failed. No retries permitted until 2026-07-16 17:36:21.365205801 +0000 UTC m=+163.851498787 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls") pod "dns-default-m24mr" (UID: "e4221d61-f6c0-43bd-b6ac-38739ab0097e") : secret "dns-default-metrics-tls" not found Jul 16 17:35:18.379683 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:18.379648 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:18.422715 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:18.422685 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" event={"ID":"6c7131e0-7ce7-42a3-8707-b8af0419d0f7","Type":"ContainerStarted","Data":"9fb46b9e955039436a3a66ec78c70e6d06d2a4153937b1677448a843c0c9c824"} Jul 16 17:35:18.440383 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:18.440355 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:18.440535 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:18.440477 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" podStartSLOduration=1.32065286 podStartE2EDuration="3.440461027s" totalImagesPullingTime="2.119808167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:18.438620702 +0000 UTC m=+100.924913707" watchObservedRunningTime="2026-07-16 17:35:18.440461027 +0000 UTC m=+100.926754033" Jul 16 17:35:18.998058 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:18.998008 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tcfbb"] Jul 16 17:35:19.000012 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:18.999995 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.007518 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.007498 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 17:35:19.008273 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.008254 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 17:35:19.008358 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.008254 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 17:35:19.008705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.008686 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 17:35:19.008813 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.008711 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lbz87\"" Jul 16 17:35:19.022732 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.022708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tcfbb"] Jul 16 17:35:19.049163 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.049134 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:19.079519 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.079493 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/df77e394-be0a-46a6-b2cc-90ff91090c9d-crio-socket\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.079641 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.079539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/df77e394-be0a-46a6-b2cc-90ff91090c9d-data-volume\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.079641 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.079558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.079641 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.079579 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl824\" (UniqueName: \"kubernetes.io/projected/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-api-access-kl824\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.079641 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.079600 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.111403 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.111363 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:19.180042 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.179992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.180202 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.180096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/df77e394-be0a-46a6-b2cc-90ff91090c9d-crio-socket\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.180202 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.180109 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:35:19.180202 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.180132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/df77e394-be0a-46a6-b2cc-90ff91090c9d-data-volume\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.180202 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.180147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.180202 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.180177 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:19.680160602 +0000 UTC m=+102.166453585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : configmap "kube-rbac-proxy" not found Jul 16 17:35:19.180202 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.180197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl824\" (UniqueName: \"kubernetes.io/projected/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-api-access-kl824\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.180451 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.180218 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/df77e394-be0a-46a6-b2cc-90ff91090c9d-crio-socket\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.180451 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.180251 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:35:19.180451 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.180305 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:19.680291173 +0000 UTC m=+102.166584172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : secret "insights-runtime-extractor-tls" not found Jul 16 17:35:19.180571 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.180503 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/df77e394-be0a-46a6-b2cc-90ff91090c9d-data-volume\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.188779 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.188760 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl824\" (UniqueName: \"kubernetes.io/projected/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-api-access-kl824\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.282062 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.281446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:19.282062 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.281613 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:35:19.282062 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.281679 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls podName:3a732f00-b481-4210-8a1c-fba79fe5df59 nodeName:}" failed. No retries permitted until 2026-07-16 17:35:23.281662752 +0000 UTC m=+105.767955736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls") pod "cluster-samples-operator-65db858894-svmmr" (UID: "3a732f00-b481-4210-8a1c-fba79fe5df59") : secret "samples-operator-tls" not found Jul 16 17:35:19.329198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.329159 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:19.401709 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.401676 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:19.480198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.480167 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:19.685522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.685433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.685522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:19.685482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:19.685722 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.685568 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:35:19.685722 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.685586 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:35:19.685722 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.685632 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:20.685617303 +0000 UTC m=+103.171910287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : secret "insights-runtime-extractor-tls" not found Jul 16 17:35:19.685722 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:19.685647 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:20.685639993 +0000 UTC m=+103.171932976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : configmap "kube-rbac-proxy" not found Jul 16 17:35:20.354693 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:20.354665 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b87sw" Jul 16 17:35:20.693840 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:20.693753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:20.693840 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:20.693796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:20.694297 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:20.693890 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:35:20.694297 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:20.693916 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:35:20.694297 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:20.693977 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:22.693957301 +0000 UTC m=+105.180250303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : secret "insights-runtime-extractor-tls" not found Jul 16 17:35:20.694297 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:20.693992 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:22.693984915 +0000 UTC m=+105.180277898 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : configmap "kube-rbac-proxy" not found Jul 16 17:35:20.789206 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:20.789181 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-72cjz_ddb47c98-02dd-4f7d-8830-d9f56f73f9e3/dns-node-resolver/0.log" Jul 16 17:35:21.989319 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:21.989285 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rsl6_1a335436-5134-4780-95ff-b3d182fb2969/node-ca/0.log" Jul 16 17:35:22.709989 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:22.709952 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:22.710206 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:22.709997 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:22.710206 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:22.710127 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:35:22.710206 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:22.710163 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:35:22.710206 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:22.710195 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:26.71017889 +0000 UTC m=+109.196471874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : secret "insights-runtime-extractor-tls" not found Jul 16 17:35:22.710348 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:22.710227 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:26.710213463 +0000 UTC m=+109.196506446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : configmap "kube-rbac-proxy" not found Jul 16 17:35:22.788718 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:22.788687 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dj4t_3615bce5-3d4d-41a5-b5dc-4e460c50ba80/serve-healthcheck-canary/0.log" Jul 16 17:35:22.935692 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:22.935651 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:22.997001 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:22.996953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:23.171950 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:23.171916 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:23.314410 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:23.314320 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:23.314554 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:23.314464 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:35:23.314554 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:23.314523 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls podName:3a732f00-b481-4210-8a1c-fba79fe5df59 nodeName:}" failed. No retries permitted until 2026-07-16 17:35:31.31450972 +0000 UTC m=+113.800802703 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls") pod "cluster-samples-operator-65db858894-svmmr" (UID: "3a732f00-b481-4210-8a1c-fba79fe5df59") : secret "samples-operator-tls" not found Jul 16 17:35:23.652649 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:23.652569 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:23.758773 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:23.758737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:24.984549 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:24.984512 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:25.084747 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.084712 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:25.258068 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.257988 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-2d5z6"] Jul 16 17:35:25.260564 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.260549 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.264438 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.264414 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 16 17:35:25.264565 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.264519 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 16 17:35:25.265442 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.265423 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 16 17:35:25.265564 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.265454 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-t4qsw\"" Jul 16 17:35:25.265685 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.265671 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 16 17:35:25.270963 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.270942 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-2d5z6"] Jul 16 17:35:25.297761 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.297739 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:25.329746 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.329723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c292d481-2ff9-4ef4-babc-4aafde90b111-signing-cabundle\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.329839 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.329792 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q4ph\" (UniqueName: \"kubernetes.io/projected/c292d481-2ff9-4ef4-babc-4aafde90b111-kube-api-access-4q4ph\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.329877 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.329858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c292d481-2ff9-4ef4-babc-4aafde90b111-signing-key\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.363543 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.363499 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:25.430644 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.430613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q4ph\" (UniqueName: \"kubernetes.io/projected/c292d481-2ff9-4ef4-babc-4aafde90b111-kube-api-access-4q4ph\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.430764 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.430681 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c292d481-2ff9-4ef4-babc-4aafde90b111-signing-key\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.430764 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.430708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c292d481-2ff9-4ef4-babc-4aafde90b111-signing-cabundle\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.431429 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.431405 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c292d481-2ff9-4ef4-babc-4aafde90b111-signing-cabundle\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.433276 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.433256 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c292d481-2ff9-4ef4-babc-4aafde90b111-signing-key\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.440503 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.440484 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q4ph\" (UniqueName: \"kubernetes.io/projected/c292d481-2ff9-4ef4-babc-4aafde90b111-kube-api-access-4q4ph\") pod \"service-ca-79f68b9f4f-2d5z6\" (UID: \"c292d481-2ff9-4ef4-babc-4aafde90b111\") " pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.568856 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.568783 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" Jul 16 17:35:25.687870 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:25.687841 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-2d5z6"] Jul 16 17:35:25.690611 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:35:25.690583 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc292d481_2ff9_4ef4_babc_4aafde90b111.slice/crio-0efe0ade44bba8432921424e2a800ce89ecd412d1b91d25c3102b2faed2e8e56 WatchSource:0}: Error finding container 0efe0ade44bba8432921424e2a800ce89ecd412d1b91d25c3102b2faed2e8e56: Status 404 returned error can't find the container with id 0efe0ade44bba8432921424e2a800ce89ecd412d1b91d25c3102b2faed2e8e56 Jul 16 17:35:26.442440 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:26.442400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" event={"ID":"c292d481-2ff9-4ef4-babc-4aafde90b111","Type":"ContainerStarted","Data":"0efe0ade44bba8432921424e2a800ce89ecd412d1b91d25c3102b2faed2e8e56"} Jul 16 17:35:26.527125 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:26.527092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:26.743409 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:26.743378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:26.743554 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:26.743486 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:26.743554 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:26.743500 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:35:26.743622 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:26.743580 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:34.743561465 +0000 UTC m=+117.229854467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : configmap "kube-rbac-proxy" not found Jul 16 17:35:26.743622 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:26.743586 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:35:26.743703 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:26.743630 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:34.74361936 +0000 UTC m=+117.229912343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : secret "insights-runtime-extractor-tls" not found Jul 16 17:35:28.449501 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:28.449412 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" event={"ID":"c292d481-2ff9-4ef4-babc-4aafde90b111","Type":"ContainerStarted","Data":"2b35ab10149294a34f54f786959c0d1a6f6591f8b2281f3ee27a6a8a251f0e64"} Jul 16 17:35:28.465846 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:28.465796 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-2d5z6" podStartSLOduration=1.176845088 podStartE2EDuration="3.465779556s" totalImagesPullingTime="2.288934468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:28.465212015 +0000 UTC m=+110.951505020" watchObservedRunningTime="2026-07-16 17:35:28.465779556 +0000 UTC m=+110.952072561" Jul 16 17:35:28.884775 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:28.884737 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:28.989897 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:28.989865 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:29.051867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:29.051821 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:29.181956 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:29.181855 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:29.282660 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:29.282623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:29.395132 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:29.395096 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:29.586522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:29.586447 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:30.383479 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:30.383450 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:30.745490 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:30.745459 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:30.994275 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:30.994242 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:31.185358 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.185265 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:31.384212 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.384182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:31.386776 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.386746 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a732f00-b481-4210-8a1c-fba79fe5df59-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-svmmr\" (UID: \"3a732f00-b481-4210-8a1c-fba79fe5df59\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:31.390210 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.390184 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:31.460659 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.460581 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" Jul 16 17:35:31.591004 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.590973 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr"] Jul 16 17:35:31.598017 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:31.597988 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:32.120132 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:32.120096 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:32.461927 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:32.461901 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" event={"ID":"3a732f00-b481-4210-8a1c-fba79fe5df59","Type":"ContainerStarted","Data":"18e64e5898cdd59d0c47020afb6cb759c92a86ed03e809737aeed3207993bc79"} Jul 16 17:35:32.946222 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:32.946121 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:33.042974 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:33.042944 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:33.192770 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:33.192723 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:33.380486 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:33.380452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:33.785118 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:33.785085 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:34.387560 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.387518 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:34.469492 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.469460 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" event={"ID":"3a732f00-b481-4210-8a1c-fba79fe5df59","Type":"ContainerStarted","Data":"3a6f6d523b7954aaf3f040f4564992ab04ac113afa5bdcf0eca402a2880a5ba5"} Jul 16 17:35:34.469625 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.469508 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" event={"ID":"3a732f00-b481-4210-8a1c-fba79fe5df59","Type":"ContainerStarted","Data":"1c67d3dcd62e669fe3998566cd73a608ef5ba0c18396bb9f8ab4ffcfdeffed2d"} Jul 16 17:35:34.489781 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.489731 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-svmmr" podStartSLOduration=17.217158224 podStartE2EDuration="19.489716296s" totalImagesPullingTime="2.272558072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:34.488363475 +0000 UTC m=+116.974656481" watchObservedRunningTime="2026-07-16 17:35:34.489716296 +0000 UTC m=+116.976009300" Jul 16 17:35:34.702676 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.702587 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:34.803998 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.803954 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:34.814067 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.814039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:34.814148 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.814107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:34.814148 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:34.814124 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:35:34.814251 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:35:34.814181 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls podName:df77e394-be0a-46a6-b2cc-90ff91090c9d nodeName:}" failed. No retries permitted until 2026-07-16 17:35:50.814165004 +0000 UTC m=+133.300457999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tcfbb" (UID: "df77e394-be0a-46a6-b2cc-90ff91090c9d") : secret "insights-runtime-extractor-tls" not found Jul 16 17:35:34.814594 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:34.814576 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/df77e394-be0a-46a6-b2cc-90ff91090c9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:35.186883 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:35.186845 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:35.398953 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:35.398907 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:35.791638 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:35.791605 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:36.380482 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:36.380443 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:36.795645 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:36.795612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:37.183757 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:37.183667 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:38.075671 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:38.075637 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:38.135515 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:38.135469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:38.588677 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:38.588635 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:38.783999 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:38.783967 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:39.227406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:39.227370 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:39.381695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:39.381667 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:39.607068 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:39.606958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:41.514652 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:41.514614 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:41.613924 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:41.613895 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:41.730459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:41.730430 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:41.809633 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:41.809541 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:41.983987 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:41.983951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:42.821220 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:42.821191 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:42.887046 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:42.887000 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:43.189973 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:43.189895 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:43.402144 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:43.402109 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:43.502823 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:43.502792 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:43.603237 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:43.603206 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:43.812004 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:43.811916 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:43.894941 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:43.894914 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:46.681342 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:46.681310 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:46.912802 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:46.912767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:35:46.915805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:46.915783 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84b3a525-e62d-44eb-b93f-2c8d1873420c-metrics-certs\") pod \"network-metrics-daemon-nm4p2\" (UID: \"84b3a525-e62d-44eb-b93f-2c8d1873420c\") " pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:35:47.074841 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:47.074804 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9hcv\"" Jul 16 17:35:47.082728 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:47.082703 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nm4p2" Jul 16 17:35:47.212086 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:47.211953 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nm4p2"] Jul 16 17:35:47.214432 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:35:47.214407 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b3a525_e62d_44eb_b93f_2c8d1873420c.slice/crio-759ac0a581527ab7932c974cdebc317c8824020886c1d6314927da5321360f25 WatchSource:0}: Error finding container 759ac0a581527ab7932c974cdebc317c8824020886c1d6314927da5321360f25: Status 404 returned error can't find the container with id 759ac0a581527ab7932c974cdebc317c8824020886c1d6314927da5321360f25 Jul 16 17:35:47.507914 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:47.507878 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nm4p2" event={"ID":"84b3a525-e62d-44eb-b93f-2c8d1873420c","Type":"ContainerStarted","Data":"759ac0a581527ab7932c974cdebc317c8824020886c1d6314927da5321360f25"} Jul 16 17:35:47.557574 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:47.557534 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:49.515530 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:49.515489 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nm4p2" event={"ID":"84b3a525-e62d-44eb-b93f-2c8d1873420c","Type":"ContainerStarted","Data":"eee360d0bf4315b805c37c2ec82298979ebcc342df8a9bdaa787bb9a63238e7f"} Jul 16 17:35:49.515929 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:49.515545 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nm4p2" event={"ID":"84b3a525-e62d-44eb-b93f-2c8d1873420c","Type":"ContainerStarted","Data":"4dc62ca594fba3c9795d1258421c57da0ad1ce900c5e46f6f61c2c564dd064ec"} Jul 16 17:35:49.533771 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:49.533725 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nm4p2" podStartSLOduration=129.63855367 podStartE2EDuration="2m11.533709274s" totalImagesPullingTime="1.895155604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:49.531932913 +0000 UTC m=+132.018225919" watchObservedRunningTime="2026-07-16 17:35:49.533709274 +0000 UTC m=+132.020002278" Jul 16 17:35:50.447452 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:50.447416 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:50.842455 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:50.842423 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:50.844734 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:50.844711 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/df77e394-be0a-46a6-b2cc-90ff91090c9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tcfbb\" (UID: \"df77e394-be0a-46a6-b2cc-90ff91090c9d\") " pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:50.900375 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:50.900330 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:50.979718 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:50.979670 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:51.037501 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.037458 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:51.112190 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.112112 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lbz87\"" Jul 16 17:35:51.120113 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.120082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tcfbb" Jul 16 17:35:51.247581 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.247554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tcfbb"] Jul 16 17:35:51.250532 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:35:51.250508 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf77e394_be0a_46a6_b2cc_90ff91090c9d.slice/crio-c8b2d3ff796b18f58ef9174d499c5992f2167c1532609619a0dfb86291deb770 WatchSource:0}: Error finding container c8b2d3ff796b18f58ef9174d499c5992f2167c1532609619a0dfb86291deb770: Status 404 returned error can't find the container with id c8b2d3ff796b18f58ef9174d499c5992f2167c1532609619a0dfb86291deb770 Jul 16 17:35:51.393414 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.393335 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5"] Jul 16 17:35:51.398878 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.398852 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.403652 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.403632 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 17:35:51.411891 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.411872 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 17:35:51.412176 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.412160 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 17:35:51.412220 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.412173 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 17:35:51.421541 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.421520 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5"] Jul 16 17:35:51.437330 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.437309 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf"] Jul 16 17:35:51.440967 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.440952 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.443926 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.443900 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 16 17:35:51.444015 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.443904 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ts8ds\"" Jul 16 17:35:51.446994 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.446973 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dc0d501-baf5-46b2-be69-c2f87a805350-tmp\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.447121 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.447064 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6dc0d501-baf5-46b2-be69-c2f87a805350-klusterlet-config\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.447121 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.447115 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmx6s\" (UniqueName: \"kubernetes.io/projected/6dc0d501-baf5-46b2-be69-c2f87a805350-kube-api-access-jmx6s\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.460274 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.460250 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf"] Jul 16 17:35:51.474130 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.474102 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:51.522126 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.522096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcfbb" event={"ID":"df77e394-be0a-46a6-b2cc-90ff91090c9d","Type":"ContainerStarted","Data":"c8b2d3ff796b18f58ef9174d499c5992f2167c1532609619a0dfb86291deb770"} Jul 16 17:35:51.547608 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.547584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6dc0d501-baf5-46b2-be69-c2f87a805350-klusterlet-config\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.547752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.547626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ffe24a63-abea-4272-9e65-19d9e67d3c65-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-895f949c6-8cskf\" (UID: \"ffe24a63-abea-4272-9e65-19d9e67d3c65\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.547752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.547660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmx6s\" (UniqueName: \"kubernetes.io/projected/6dc0d501-baf5-46b2-be69-c2f87a805350-kube-api-access-jmx6s\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.547752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.547702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcc72\" (UniqueName: \"kubernetes.io/projected/ffe24a63-abea-4272-9e65-19d9e67d3c65-kube-api-access-rcc72\") pod \"managed-serviceaccount-addon-agent-895f949c6-8cskf\" (UID: \"ffe24a63-abea-4272-9e65-19d9e67d3c65\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.547752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.547743 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dc0d501-baf5-46b2-be69-c2f87a805350-tmp\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.548154 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.548136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6dc0d501-baf5-46b2-be69-c2f87a805350-tmp\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.550522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.550503 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6dc0d501-baf5-46b2-be69-c2f87a805350-klusterlet-config\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.592993 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.592963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmx6s\" (UniqueName: \"kubernetes.io/projected/6dc0d501-baf5-46b2-be69-c2f87a805350-kube-api-access-jmx6s\") pod \"klusterlet-addon-workmgr-76bb794fb7-rf7t5\" (UID: \"6dc0d501-baf5-46b2-be69-c2f87a805350\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.648104 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.647985 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:51.648104 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.648091 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ffe24a63-abea-4272-9e65-19d9e67d3c65-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-895f949c6-8cskf\" (UID: \"ffe24a63-abea-4272-9e65-19d9e67d3c65\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.648263 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.648145 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcc72\" (UniqueName: \"kubernetes.io/projected/ffe24a63-abea-4272-9e65-19d9e67d3c65-kube-api-access-rcc72\") pod \"managed-serviceaccount-addon-agent-895f949c6-8cskf\" (UID: \"ffe24a63-abea-4272-9e65-19d9e67d3c65\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.650535 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.650515 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ffe24a63-abea-4272-9e65-19d9e67d3c65-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-895f949c6-8cskf\" (UID: \"ffe24a63-abea-4272-9e65-19d9e67d3c65\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.665980 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.665959 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcc72\" (UniqueName: \"kubernetes.io/projected/ffe24a63-abea-4272-9e65-19d9e67d3c65-kube-api-access-rcc72\") pod \"managed-serviceaccount-addon-agent-895f949c6-8cskf\" (UID: \"ffe24a63-abea-4272-9e65-19d9e67d3c65\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.708069 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.708046 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:51.758262 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.758228 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" Jul 16 17:35:51.811379 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.811311 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:51.919314 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.919214 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5"] Jul 16 17:35:51.975561 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:51.975532 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf"] Jul 16 17:35:51.980151 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:35:51.980123 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffe24a63_abea_4272_9e65_19d9e67d3c65.slice/crio-6d435b6e147887d8dcbbb2b676c09603a78effb8ae653224b083c60f3eda3959 WatchSource:0}: Error finding container 6d435b6e147887d8dcbbb2b676c09603a78effb8ae653224b083c60f3eda3959: Status 404 returned error can't find the container with id 6d435b6e147887d8dcbbb2b676c09603a78effb8ae653224b083c60f3eda3959 Jul 16 17:35:52.527367 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:52.527328 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" event={"ID":"6dc0d501-baf5-46b2-be69-c2f87a805350","Type":"ContainerStarted","Data":"dcb49055622fd936c2041c9a41a0780c3f369b07a90c576fff54652cb918a03d"} Jul 16 17:35:52.528610 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:52.528576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" event={"ID":"ffe24a63-abea-4272-9e65-19d9e67d3c65","Type":"ContainerStarted","Data":"6d435b6e147887d8dcbbb2b676c09603a78effb8ae653224b083c60f3eda3959"} Jul 16 17:35:53.820988 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.820951 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:53.824946 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.824921 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-649ff4df44-w7thm"] Jul 16 17:35:53.830743 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.829298 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.834086 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.834015 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fb559\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.836296 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.836496 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.836502 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.836722 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.836755 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.836941 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 17:35:53.837780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.837170 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 17:35:53.848927 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.848580 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 17:35:53.848927 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.848743 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-649ff4df44-w7thm"] Jul 16 17:35:53.867981 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.867942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-serving-cert\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.868134 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.867986 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-config\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.868199 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.868135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-service-ca\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.868199 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.868182 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb744\" (UniqueName: \"kubernetes.io/projected/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-kube-api-access-zb744\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.868307 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.868238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-trusted-ca-bundle\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.868354 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.868301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-oauth-config\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.868354 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.868328 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-oauth-serving-cert\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.885167 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.885136 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:53.969535 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-oauth-config\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.969535 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-oauth-serving-cert\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.969766 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-serving-cert\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.969766 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-config\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.969766 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969677 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-service-ca\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.969766 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb744\" (UniqueName: \"kubernetes.io/projected/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-kube-api-access-zb744\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.969766 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.969741 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-trusted-ca-bundle\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.970724 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.970653 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-trusted-ca-bundle\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.970724 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.970694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-oauth-serving-cert\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.970923 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.970786 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-config\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.970923 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.970828 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-service-ca\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.977148 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.977106 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-oauth-config\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.977263 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.977240 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-serving-cert\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:53.989332 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:53.989237 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb744\" (UniqueName: \"kubernetes.io/projected/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-kube-api-access-zb744\") pod \"console-649ff4df44-w7thm\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:54.152380 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:54.152127 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:35:55.122503 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:55.122469 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:55.224150 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:55.224098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:58.963147 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:58.963121 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-649ff4df44-w7thm"] Jul 16 17:35:58.967098 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:35:58.967070 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37673956_6dbc_422b_8f8c_5a3dd4dab3a3.slice/crio-fb775b5de6d805522f4d4c749f70109af955939eab9b28f4064009731022e4bb WatchSource:0}: Error finding container fb775b5de6d805522f4d4c749f70109af955939eab9b28f4064009731022e4bb: Status 404 returned error can't find the container with id fb775b5de6d805522f4d4c749f70109af955939eab9b28f4064009731022e4bb Jul 16 17:35:59.185152 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.185047 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:59.252814 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.252776 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:59.560204 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.560166 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcfbb" event={"ID":"df77e394-be0a-46a6-b2cc-90ff91090c9d","Type":"ContainerStarted","Data":"83e8d80326cb7129dce34fb244ffde4e696a60bb5d0dbd2bba77354a2b74fd25"} Jul 16 17:35:59.562161 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.562101 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" event={"ID":"ffe24a63-abea-4272-9e65-19d9e67d3c65","Type":"ContainerStarted","Data":"24d5ef49b3270fece9f44de04277cc922c91881b5fabf22732b136a40261743c"} Jul 16 17:35:59.563892 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.563865 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649ff4df44-w7thm" event={"ID":"37673956-6dbc-422b-8f8c-5a3dd4dab3a3","Type":"ContainerStarted","Data":"fb775b5de6d805522f4d4c749f70109af955939eab9b28f4064009731022e4bb"} Jul 16 17:35:59.566666 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.566640 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" event={"ID":"6dc0d501-baf5-46b2-be69-c2f87a805350","Type":"ContainerStarted","Data":"f7bb818477d1bd04c3ea11893e725be9ac3164647c5508253d73e8f5cec342ec"} Jul 16 17:35:59.566965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.566948 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:59.568742 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.568722 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" Jul 16 17:35:59.601763 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.601708 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bb794fb7-rf7t5" podStartSLOduration=1.632093136 podStartE2EDuration="8.601687485s" totalImagesPullingTime="6.969594349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:59.600052289 +0000 UTC m=+142.086345293" watchObservedRunningTime="2026-07-16 17:35:59.601687485 +0000 UTC m=+142.087980491" Jul 16 17:35:59.602214 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.602175 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-895f949c6-8cskf" podStartSLOduration=1.7183154200000001 podStartE2EDuration="8.602161933s" totalImagesPullingTime="6.883846513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:59.580155997 +0000 UTC m=+142.066449003" watchObservedRunningTime="2026-07-16 17:35:59.602161933 +0000 UTC m=+142.088454920" Jul 16 17:35:59.653112 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.652998 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:59.728314 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:35:59.728261 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:00.122747 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:00.122709 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:01.213140 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:01.213098 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:01.270526 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:01.270495 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:01.683705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:01.683613 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:01.747966 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:01.747933 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:04.588449 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:04.588409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcfbb" event={"ID":"df77e394-be0a-46a6-b2cc-90ff91090c9d","Type":"ContainerStarted","Data":"7819e1d44d4047d25ae8e77c4e9bd2f8764c337d556b86b94cb73564fb23d279"} Jul 16 17:36:04.590004 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:04.589976 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649ff4df44-w7thm" event={"ID":"37673956-6dbc-422b-8f8c-5a3dd4dab3a3","Type":"ContainerStarted","Data":"6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740"} Jul 16 17:36:04.610669 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:04.610625 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-649ff4df44-w7thm" podStartSLOduration=7.034703239 podStartE2EDuration="11.610610409s" totalImagesPullingTime="4.57590717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:04.609195478 +0000 UTC m=+147.095488485" watchObservedRunningTime="2026-07-16 17:36:04.610610409 +0000 UTC m=+147.096903413" Jul 16 17:36:06.299704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:06.299650 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:06.357747 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:06.357702 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:07.601988 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:07.601944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tcfbb" event={"ID":"df77e394-be0a-46a6-b2cc-90ff91090c9d","Type":"ContainerStarted","Data":"54bbd04e0c79b0106913c85575782557a4848409e910f49db5d7bc5bccebddc8"} Jul 16 17:36:07.621406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:07.621357 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tcfbb" podStartSLOduration=34.012244216 podStartE2EDuration="49.621342685s" totalImagesPullingTime="15.609098469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:35:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:07.620196266 +0000 UTC m=+150.106489271" watchObservedRunningTime="2026-07-16 17:36:07.621342685 +0000 UTC m=+150.107635680" Jul 16 17:36:12.123599 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:12.123561 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:12.183805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:12.183772 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:13.390690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:13.390655 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:13.464044 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:13.464002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:14.069509 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.069474 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f5f9c65dc-lk5lx"] Jul 16 17:36:14.069729 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:36:14.069707 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" podUID="e4ea9957-783c-4c4a-a665-7027a9a8caae" Jul 16 17:36:14.152944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.152903 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:36:14.153098 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.152984 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:36:14.157695 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.157674 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:36:14.169490 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.169452 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:14.231846 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.231808 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:14.334013 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.333921 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:14.436533 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.436502 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:14.623571 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.623495 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:36:14.627545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.627521 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:36:14.629360 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.629343 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:36:14.765235 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765201 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-installation-pull-secrets\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765235 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765239 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq6d6\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-kube-api-access-vq6d6\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765447 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765282 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-bound-sa-token\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765447 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765302 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-certificates\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765447 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765326 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-image-registry-private-configuration\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765447 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765360 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4ea9957-783c-4c4a-a665-7027a9a8caae-ca-trust-extracted\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765447 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765397 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-trusted-ca\") pod \"e4ea9957-783c-4c4a-a665-7027a9a8caae\" (UID: \"e4ea9957-783c-4c4a-a665-7027a9a8caae\") " Jul 16 17:36:14.765710 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.765674 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:36:14.766207 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.766054 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:36:14.766411 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.766384 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ea9957-783c-4c4a-a665-7027a9a8caae-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:36:14.767822 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.767799 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:36:14.767947 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.767929 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:36:14.768038 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.768008 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-kube-api-access-vq6d6" (OuterVolumeSpecName: "kube-api-access-vq6d6") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "kube-api-access-vq6d6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:36:14.768129 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.768061 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e4ea9957-783c-4c4a-a665-7027a9a8caae" (UID: "e4ea9957-783c-4c4a-a665-7027a9a8caae"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:36:14.866523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866489 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-trusted-ca\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:14.866523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866518 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-installation-pull-secrets\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:14.866523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866527 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq6d6\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-kube-api-access-vq6d6\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:14.866523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866536 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-bound-sa-token\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:14.866781 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866545 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-certificates\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:14.866781 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866553 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4ea9957-783c-4c4a-a665-7027a9a8caae-image-registry-private-configuration\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:14.866781 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:14.866561 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4ea9957-783c-4c4a-a665-7027a9a8caae-ca-trust-extracted\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:15.626921 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:15.626883 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5f9c65dc-lk5lx" Jul 16 17:36:15.665525 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:15.665493 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f5f9c65dc-lk5lx"] Jul 16 17:36:15.670580 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:15.670555 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f5f9c65dc-lk5lx"] Jul 16 17:36:15.773579 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:15.773553 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4ea9957-783c-4c4a-a665-7027a9a8caae-registry-tls\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:15.811910 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:15.811875 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.062828 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.062787 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ea9957-783c-4c4a-a665-7027a9a8caae" path="/var/lib/kubelet/pods/e4ea9957-783c-4c4a-a665-7027a9a8caae/volumes" Jul 16 17:36:16.126338 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.126298 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.151360 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.151329 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp"] Jul 16 17:36:16.154313 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.154295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.157198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.156873 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 17:36:16.157198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.156896 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 17:36:16.157198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.156958 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 17:36:16.157198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.156900 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 17:36:16.157198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.157151 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 17:36:16.157198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.157164 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kcl6b\"" Jul 16 17:36:16.175110 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.175086 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp"] Jul 16 17:36:16.176372 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.176349 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-826js"] Jul 16 17:36:16.178805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.178785 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.181230 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.181209 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 16 17:36:16.181439 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.181418 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gknf7\"" Jul 16 17:36:16.181517 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.181278 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 16 17:36:16.181690 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.181665 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 16 17:36:16.186082 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.186060 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b82tg"] Jul 16 17:36:16.188634 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.188615 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.190960 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.190941 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 17:36:16.191364 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.191351 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 17:36:16.191524 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.191506 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zrzhn\"" Jul 16 17:36:16.191687 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.191373 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 17:36:16.192568 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.192546 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-826js"] Jul 16 17:36:16.215992 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.215960 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.277727 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-accelerators-collector-config\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.277727 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhkkb\" (UniqueName: \"kubernetes.io/projected/a25478d6-c0b0-45eb-8bf4-0aa93160747c-kube-api-access-fhkkb\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.277954 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-textfile\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.277954 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277774 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-tls\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.277954 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-wtmp\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.277954 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277893 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.277954 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.277989 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25478d6-c0b0-45eb-8bf4-0aa93160747c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278076 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278107 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-sys\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278133 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.278179 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278167 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-root\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.278370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/485e738c-fcf1-4b9a-a462-d59721e1bcba-metrics-client-ca\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.278370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klvpc\" (UniqueName: \"kubernetes.io/projected/485e738c-fcf1-4b9a-a462-d59721e1bcba-kube-api-access-klvpc\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.278370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.278370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278315 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42dv\" (UniqueName: \"kubernetes.io/projected/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-api-access-b42dv\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.278370 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.278336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.285437 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.285393 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.379400 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-tls\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379400 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-wtmp\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379488 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25478d6-c0b0-45eb-8bf4-0aa93160747c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379554 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-wtmp\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379607 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-sys\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-root\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379663 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/485e738c-fcf1-4b9a-a462-d59721e1bcba-metrics-client-ca\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:36:16.379669 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klvpc\" (UniqueName: \"kubernetes.io/projected/485e738c-fcf1-4b9a-a462-d59721e1bcba-kube-api-access-klvpc\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:36:16.379728 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-tls podName:a25478d6-c0b0-45eb-8bf4-0aa93160747c nodeName:}" failed. No retries permitted until 2026-07-16 17:36:16.879708391 +0000 UTC m=+159.366001378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-rt4zp" (UID: "a25478d6-c0b0-45eb-8bf4-0aa93160747c") : secret "openshift-state-metrics-tls" not found Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42dv\" (UniqueName: \"kubernetes.io/projected/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-api-access-b42dv\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-accelerators-collector-config\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhkkb\" (UniqueName: \"kubernetes.io/projected/a25478d6-c0b0-45eb-8bf4-0aa93160747c-kube-api-access-fhkkb\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.379991 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.379934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-textfile\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.380729 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.380040 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-sys\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.380729 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.380248 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25478d6-c0b0-45eb-8bf4-0aa93160747c-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.380729 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.380329 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-textfile\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.381502 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.381476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.381602 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.381558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/485e738c-fcf1-4b9a-a462-d59721e1bcba-root\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.382244 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382212 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-accelerators-collector-config\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.382244 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382238 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.382521 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382499 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.382610 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382591 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/485e738c-fcf1-4b9a-a462-d59721e1bcba-node-exporter-tls\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.382763 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382737 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.382852 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382797 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/485e738c-fcf1-4b9a-a462-d59721e1bcba-metrics-client-ca\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.382916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382893 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.382962 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.382929 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.383283 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.383263 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.388927 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.388897 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.391711 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.391687 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klvpc\" (UniqueName: \"kubernetes.io/projected/485e738c-fcf1-4b9a-a462-d59721e1bcba-kube-api-access-klvpc\") pod \"node-exporter-b82tg\" (UID: \"485e738c-fcf1-4b9a-a462-d59721e1bcba\") " pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.391996 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.391967 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42dv\" (UniqueName: \"kubernetes.io/projected/1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533-kube-api-access-b42dv\") pod \"kube-state-metrics-785778ff74-826js\" (UID: \"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.392310 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.392288 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhkkb\" (UniqueName: \"kubernetes.io/projected/a25478d6-c0b0-45eb-8bf4-0aa93160747c-kube-api-access-fhkkb\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.484434 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:36:16.484387 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-m24mr" podUID="e4221d61-f6c0-43bd-b6ac-38739ab0097e" Jul 16 17:36:16.489510 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.489476 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.490237 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.490220 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" Jul 16 17:36:16.499776 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.499757 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b82tg" Jul 16 17:36:16.509797 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:36:16.509772 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485e738c_fcf1_4b9a_a462_d59721e1bcba.slice/crio-dbc6c37e4a72d7b0c6cbb75ca10c1b4f6da4288cef6059e51512a5528fe7d09a WatchSource:0}: Error finding container dbc6c37e4a72d7b0c6cbb75ca10c1b4f6da4288cef6059e51512a5528fe7d09a: Status 404 returned error can't find the container with id dbc6c37e4a72d7b0c6cbb75ca10c1b4f6da4288cef6059e51512a5528fe7d09a Jul 16 17:36:16.587667 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.587619 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.626461 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.626436 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-826js"] Jul 16 17:36:16.628713 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:36:16.628687 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c45d6a7_bcc0_4f59_bb4a_3cc3629cb533.slice/crio-e4a06bba59b8fbe453988f0b5b26425dca1e9c51c130093dc158f05d36832f0b WatchSource:0}: Error finding container e4a06bba59b8fbe453988f0b5b26425dca1e9c51c130093dc158f05d36832f0b: Status 404 returned error can't find the container with id e4a06bba59b8fbe453988f0b5b26425dca1e9c51c130093dc158f05d36832f0b Jul 16 17:36:16.632258 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.632229 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b82tg" event={"ID":"485e738c-fcf1-4b9a-a462-d59721e1bcba","Type":"ContainerStarted","Data":"dbc6c37e4a72d7b0c6cbb75ca10c1b4f6da4288cef6059e51512a5528fe7d09a"} Jul 16 17:36:16.632384 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.632269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m24mr" Jul 16 17:36:16.819993 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.819957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:16.883315 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.883231 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:16.885767 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:16.885701 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a25478d6-c0b0-45eb-8bf4-0aa93160747c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-rt4zp\" (UID: \"a25478d6-c0b0-45eb-8bf4-0aa93160747c\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:17.039498 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.039464 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:17.065478 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.065451 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" Jul 16 17:36:17.209444 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.209415 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp"] Jul 16 17:36:17.212266 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:36:17.212231 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25478d6_c0b0_45eb_8bf4_0aa93160747c.slice/crio-e8c545225ddb586592188704ff0862dd7b3f4e17e6e892e802a2536a83e0506f WatchSource:0}: Error finding container e8c545225ddb586592188704ff0862dd7b3f4e17e6e892e802a2536a83e0506f: Status 404 returned error can't find the container with id e8c545225ddb586592188704ff0862dd7b3f4e17e6e892e802a2536a83e0506f Jul 16 17:36:17.638882 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.638831 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b82tg" event={"ID":"485e738c-fcf1-4b9a-a462-d59721e1bcba","Type":"ContainerStarted","Data":"b4f9b4e11a22ac8e9841474a38f4e35e26400a55e2c5829ba566b9f771456ffe"} Jul 16 17:36:17.641245 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.641212 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" event={"ID":"a25478d6-c0b0-45eb-8bf4-0aa93160747c","Type":"ContainerStarted","Data":"347608b0d926825742b08de846513105079f49460d189ac7a023a95c5b2ad6b5"} Jul 16 17:36:17.641376 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.641278 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" event={"ID":"a25478d6-c0b0-45eb-8bf4-0aa93160747c","Type":"ContainerStarted","Data":"13702a1b05fab4741e1cdc582d302056b102ca4aae342a4669aa9535aff31b50"} Jul 16 17:36:17.641376 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.641305 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" event={"ID":"a25478d6-c0b0-45eb-8bf4-0aa93160747c","Type":"ContainerStarted","Data":"e8c545225ddb586592188704ff0862dd7b3f4e17e6e892e802a2536a83e0506f"} Jul 16 17:36:17.642654 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:17.642626 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" event={"ID":"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533","Type":"ContainerStarted","Data":"e4a06bba59b8fbe453988f0b5b26425dca1e9c51c130093dc158f05d36832f0b"} Jul 16 17:36:18.091677 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.091636 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:18.648248 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.648143 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" event={"ID":"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533","Type":"ContainerStarted","Data":"3dd414406856ad0870a2b2a45398053e769d2eb917c49a224ac7889855031e04"} Jul 16 17:36:18.648248 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.648214 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" event={"ID":"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533","Type":"ContainerStarted","Data":"19951875b0ae47f874dadca34b92101d05f57b455a67f74ea25a684954468699"} Jul 16 17:36:18.648248 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.648237 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" event={"ID":"1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533","Type":"ContainerStarted","Data":"d677a78ca41b5ee61f8c78da76e14003613eaa909ecade11c138971dfd87648f"} Jul 16 17:36:18.649647 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.649622 2579 generic.go:356] "Generic (PLEG): container finished" podID="485e738c-fcf1-4b9a-a462-d59721e1bcba" containerID="b4f9b4e11a22ac8e9841474a38f4e35e26400a55e2c5829ba566b9f771456ffe" exitCode=0 Jul 16 17:36:18.649747 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.649654 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b82tg" event={"ID":"485e738c-fcf1-4b9a-a462-d59721e1bcba","Type":"ContainerDied","Data":"b4f9b4e11a22ac8e9841474a38f4e35e26400a55e2c5829ba566b9f771456ffe"} Jul 16 17:36:18.668258 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:18.668206 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-826js" podStartSLOduration=1.115774731 podStartE2EDuration="2.668188589s" totalImagesPullingTime="1.552413858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:18.667675518 +0000 UTC m=+161.153968524" watchObservedRunningTime="2026-07-16 17:36:18.668188589 +0000 UTC m=+161.154481597" Jul 16 17:36:19.023813 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.023771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:19.101415 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.101372 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:19.297625 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.297589 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:19.654854 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.654761 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" event={"ID":"a25478d6-c0b0-45eb-8bf4-0aa93160747c","Type":"ContainerStarted","Data":"c2c597bf776a1528d5c0282c22f6eca1236c0ef6f3e530077a32da14c1ec9873"} Jul 16 17:36:19.656849 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.656818 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b82tg" event={"ID":"485e738c-fcf1-4b9a-a462-d59721e1bcba","Type":"ContainerStarted","Data":"06620021843b0276bb18c6c14c331d48ffc2d16d82cff8e6b7ba8b75a9af9587"} Jul 16 17:36:19.656973 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.656876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b82tg" event={"ID":"485e738c-fcf1-4b9a-a462-d59721e1bcba","Type":"ContainerStarted","Data":"30617e7400ad75ee84a2b96da5d143c2848dd19a3b59979efd8bd2d1075d95f9"} Jul 16 17:36:19.675444 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.675383 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-rt4zp" podStartSLOduration=2.031706665 podStartE2EDuration="3.6753675s" totalImagesPullingTime="1.643660835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:19.673707071 +0000 UTC m=+162.160000076" watchObservedRunningTime="2026-07-16 17:36:19.6753675 +0000 UTC m=+162.161660504" Jul 16 17:36:19.694205 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.694156 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b82tg" podStartSLOduration=2.64136469 podStartE2EDuration="3.694138896s" totalImagesPullingTime="978.268513ms" totalInitContainerRuntime="74.505693ms" isStatefulPod=true podCreationTimestamp="2026-07-16 17:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:19.692880778 +0000 UTC m=+162.179173784" watchObservedRunningTime="2026-07-16 17:36:19.694138896 +0000 UTC m=+162.180431900" Jul 16 17:36:19.818890 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:19.818862 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:20.629326 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.629292 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:20.728700 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.728672 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:20.857669 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.857632 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq"] Jul 16 17:36:20.860965 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.860935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:20.863771 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.863755 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zv949\"" Jul 16 17:36:20.865657 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.865634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:20.866339 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.866319 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 17:36:20.882944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:20.882884 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq"] Jul 16 17:36:21.020172 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.020142 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxzms\" (UniqueName: \"kubernetes.io/projected/d9e40ed8-6afe-473d-b3c8-00c98930c2a6-kube-api-access-cxzms\") pod \"monitoring-plugin-5f68df7f58-gbxbq\" (UID: \"d9e40ed8-6afe-473d-b3c8-00c98930c2a6\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.020331 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.020202 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d9e40ed8-6afe-473d-b3c8-00c98930c2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-gbxbq\" (UID: \"d9e40ed8-6afe-473d-b3c8-00c98930c2a6\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.052514 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.052485 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:21.120987 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.120948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d9e40ed8-6afe-473d-b3c8-00c98930c2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-gbxbq\" (UID: \"d9e40ed8-6afe-473d-b3c8-00c98930c2a6\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.121199 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.121069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxzms\" (UniqueName: \"kubernetes.io/projected/d9e40ed8-6afe-473d-b3c8-00c98930c2a6-kube-api-access-cxzms\") pod \"monitoring-plugin-5f68df7f58-gbxbq\" (UID: \"d9e40ed8-6afe-473d-b3c8-00c98930c2a6\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.122306 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.122268 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:21.124066 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.124041 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d9e40ed8-6afe-473d-b3c8-00c98930c2a6-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-gbxbq\" (UID: \"d9e40ed8-6afe-473d-b3c8-00c98930c2a6\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.130605 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.130578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxzms\" (UniqueName: \"kubernetes.io/projected/d9e40ed8-6afe-473d-b3c8-00c98930c2a6-kube-api-access-cxzms\") pod \"monitoring-plugin-5f68df7f58-gbxbq\" (UID: \"d9e40ed8-6afe-473d-b3c8-00c98930c2a6\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.170473 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.170398 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:21.304153 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.304126 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq"] Jul 16 17:36:21.307198 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:36:21.307150 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e40ed8_6afe_473d_b3c8_00c98930c2a6.slice/crio-c785734ec647f65c0aba274189d7171fa74bb85eb9e5d451a29b0c51526548cc WatchSource:0}: Error finding container c785734ec647f65c0aba274189d7171fa74bb85eb9e5d451a29b0c51526548cc: Status 404 returned error can't find the container with id c785734ec647f65c0aba274189d7171fa74bb85eb9e5d451a29b0c51526548cc Jul 16 17:36:21.425226 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.425109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:36:21.427504 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.427474 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4221d61-f6c0-43bd-b6ac-38739ab0097e-metrics-tls\") pod \"dns-default-m24mr\" (UID: \"e4221d61-f6c0-43bd-b6ac-38739ab0097e\") " pod="openshift-dns/dns-default-m24mr" Jul 16 17:36:21.436627 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.436606 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ntp5m\"" Jul 16 17:36:21.443909 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.443890 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m24mr" Jul 16 17:36:21.583746 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.583714 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m24mr"] Jul 16 17:36:21.586637 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:36:21.586617 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4221d61_f6c0_43bd_b6ac_38739ab0097e.slice/crio-718b547580a0a0a563f06c8fda6bb418bd1f50f7c219a1d845dd36bcbc233525 WatchSource:0}: Error finding container 718b547580a0a0a563f06c8fda6bb418bd1f50f7c219a1d845dd36bcbc233525: Status 404 returned error can't find the container with id 718b547580a0a0a563f06c8fda6bb418bd1f50f7c219a1d845dd36bcbc233525 Jul 16 17:36:21.665791 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.665749 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m24mr" event={"ID":"e4221d61-f6c0-43bd-b6ac-38739ab0097e","Type":"ContainerStarted","Data":"718b547580a0a0a563f06c8fda6bb418bd1f50f7c219a1d845dd36bcbc233525"} Jul 16 17:36:21.666825 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:21.666796 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" event={"ID":"d9e40ed8-6afe-473d-b3c8-00c98930c2a6","Type":"ContainerStarted","Data":"c785734ec647f65c0aba274189d7171fa74bb85eb9e5d451a29b0c51526548cc"} Jul 16 17:36:22.283180 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.283002 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:22.351212 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.351143 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:36:22.359799 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.359765 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.362704 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.362677 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 17:36:22.363013 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.362996 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 17:36:22.363620 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.363594 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 17:36:22.363720 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.363384 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 17:36:22.364046 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.363406 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 17:36:22.364046 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.363481 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2kgrq\"" Jul 16 17:36:22.364046 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.363334 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-du89j81hkv2f4\"" Jul 16 17:36:22.364595 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.364579 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 17:36:22.364966 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.364594 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 17:36:22.365187 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.365168 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 17:36:22.365379 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.365365 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 17:36:22.365462 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.365400 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 17:36:22.365462 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.365425 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 17:36:22.365560 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.365365 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 17:36:22.366060 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.365752 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 17:36:22.370814 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.370539 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 17:36:22.371820 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.371799 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 17:36:22.377055 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.377000 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:36:22.407414 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.407376 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:22.535312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535210 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535294 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535318 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-web-config\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535353 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535432 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535456 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535482 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535505 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8bgw\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-kube-api-access-w8bgw\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535545 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535630 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config-out\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535816 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.535939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.535848 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636567 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636567 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636823 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636823 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636756 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-web-config\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636823 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636978 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.636978 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.636956 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637101 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637101 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637101 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637247 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637106 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8bgw\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-kube-api-access-w8bgw\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637247 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637153 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637247 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637247 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637210 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637247 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637477 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637269 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637477 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637299 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config-out\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637477 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637334 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637477 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637365 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637477 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637396 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.637877 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.637805 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.638536 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.638397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.639951 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.639520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.639951 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.639711 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.640442 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.640385 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.640683 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.640659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.640760 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.640748 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.641200 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.641178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.641344 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.641321 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-web-config\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.641583 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.641559 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.642235 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.642209 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.642748 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.642722 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.642825 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.642773 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config-out\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.643727 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.643700 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.644672 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.644195 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.644672 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.644629 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.644672 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.644652 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.645428 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.645403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.645663 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.645644 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.647448 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.647427 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8bgw\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-kube-api-access-w8bgw\") pod \"prometheus-k8s-0\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:22.677266 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:22.677215 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:23.460800 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.460774 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:36:23.468148 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:36:23.468123 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode002fc7b_f4e1_4162_82d2_a0624be4ac06.slice/crio-b89d092452d32f4c84e8dec0c1de1599726368bca4b42ec5a68eb5ff52ffc325 WatchSource:0}: Error finding container b89d092452d32f4c84e8dec0c1de1599726368bca4b42ec5a68eb5ff52ffc325: Status 404 returned error can't find the container with id b89d092452d32f4c84e8dec0c1de1599726368bca4b42ec5a68eb5ff52ffc325 Jul 16 17:36:23.676468 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.676379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m24mr" event={"ID":"e4221d61-f6c0-43bd-b6ac-38739ab0097e","Type":"ContainerStarted","Data":"a711d2d2c92ef70760273572fa83f3b686724519a433f1178addb892e17cf992"} Jul 16 17:36:23.676468 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.676430 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m24mr" event={"ID":"e4221d61-f6c0-43bd-b6ac-38739ab0097e","Type":"ContainerStarted","Data":"657c0ddd1824f6a3765a420356e5eb1975aef7d6429c17aa96f855b378ae22ca"} Jul 16 17:36:23.676468 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.676459 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m24mr" Jul 16 17:36:23.677815 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.677790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" event={"ID":"d9e40ed8-6afe-473d-b3c8-00c98930c2a6","Type":"ContainerStarted","Data":"de3dcbbd7fc1da2a6ff30adeb27749902157162f6376522c96ca80690beceac2"} Jul 16 17:36:23.677944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.677839 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:23.679054 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.679008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"b89d092452d32f4c84e8dec0c1de1599726368bca4b42ec5a68eb5ff52ffc325"} Jul 16 17:36:23.682810 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.682789 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" Jul 16 17:36:23.697009 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.696960 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m24mr" podStartSLOduration=128.972922948 podStartE2EDuration="2m10.696932428s" totalImagesPullingTime="1.72400948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:23.695992937 +0000 UTC m=+166.182285939" watchObservedRunningTime="2026-07-16 17:36:23.696932428 +0000 UTC m=+166.183225432" Jul 16 17:36:23.716853 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:23.716796 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-gbxbq" podStartSLOduration=1.719075093 podStartE2EDuration="3.716781684s" totalImagesPullingTime="1.997706591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:23.715714121 +0000 UTC m=+166.202007135" watchObservedRunningTime="2026-07-16 17:36:23.716781684 +0000 UTC m=+166.203074688" Jul 16 17:36:25.689072 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:25.688940 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" exitCode=0 Jul 16 17:36:25.689072 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:25.689045 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} Jul 16 17:36:26.378311 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:26.378264 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:26.439931 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:26.439887 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:27.229960 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:27.229918 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:27.344706 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:27.344661 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:27.930490 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:27.930453 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:30.711633 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:30.711607 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} Jul 16 17:36:30.711989 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:30.711670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} Jul 16 17:36:31.691940 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:31.691904 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:32.296752 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:32.296717 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-649ff4df44-w7thm"] Jul 16 17:36:32.557368 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:32.557335 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:32.722244 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:32.722200 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} Jul 16 17:36:33.054065 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:33.053952 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:33.686137 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:33.686111 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m24mr" Jul 16 17:36:33.731359 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:33.731322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} Jul 16 17:36:33.731514 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:33.731406 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} Jul 16 17:36:33.731514 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:33.731455 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerStarted","Data":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} Jul 16 17:36:33.761635 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:33.761589 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.345766087 podStartE2EDuration="11.761566764s" totalImagesPullingTime="8.395085473s" totalInitContainerRuntime="20.715204ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:36:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:36:33.760436827 +0000 UTC m=+176.246729855" watchObservedRunningTime="2026-07-16 17:36:33.761566764 +0000 UTC m=+176.247859770" Jul 16 17:36:37.677598 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:37.677559 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:36:42.831067 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:42.831011 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:42.898217 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:42.898181 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:46.682995 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:46.682958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:46.904707 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:46.904659 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:48.788794 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:48.788757 2579 generic.go:356] "Generic (PLEG): container finished" podID="6c7131e0-7ce7-42a3-8707-b8af0419d0f7" containerID="9fb46b9e955039436a3a66ec78c70e6d06d2a4153937b1677448a843c0c9c824" exitCode=0 Jul 16 17:36:48.789192 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:48.788831 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" event={"ID":"6c7131e0-7ce7-42a3-8707-b8af0419d0f7","Type":"ContainerDied","Data":"9fb46b9e955039436a3a66ec78c70e6d06d2a4153937b1677448a843c0c9c824"} Jul 16 17:36:48.789192 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:48.789156 2579 scope.go:117] "RemoveContainer" containerID="9fb46b9e955039436a3a66ec78c70e6d06d2a4153937b1677448a843c0c9c824" Jul 16 17:36:49.794174 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:49.794141 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-stvkn" event={"ID":"6c7131e0-7ce7-42a3-8707-b8af0419d0f7","Type":"ContainerStarted","Data":"446741fc539c707667e79c4d435b30ea14686433352a08b5e924d93f61445312"} Jul 16 17:36:51.852153 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:51.852113 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:36:57.317765 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.317723 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-649ff4df44-w7thm" podUID="37673956-6dbc-422b-8f8c-5a3dd4dab3a3" containerName="console" containerID="cri-o://6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740" gracePeriod=15 Jul 16 17:36:57.555534 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.555511 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:36:57.615763 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615689 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb744\" (UniqueName: \"kubernetes.io/projected/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-kube-api-access-zb744\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.615763 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615726 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-trusted-ca-bundle\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.615763 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615755 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-config\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.615970 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615777 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-serving-cert\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.615970 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615798 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-oauth-config\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.615970 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615841 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-service-ca\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.615970 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.615883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-oauth-serving-cert\") pod \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\" (UID: \"37673956-6dbc-422b-8f8c-5a3dd4dab3a3\") " Jul 16 17:36:57.616523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.616221 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-config" (OuterVolumeSpecName: "console-config") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:36:57.616523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.616301 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:36:57.616523 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.616371 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:36:57.616638 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.616549 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:36:57.618214 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.618179 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:36:57.618312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.618241 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-kube-api-access-zb744" (OuterVolumeSpecName: "kube-api-access-zb744") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "kube-api-access-zb744". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:36:57.618312 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.618240 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "37673956-6dbc-422b-8f8c-5a3dd4dab3a3" (UID: "37673956-6dbc-422b-8f8c-5a3dd4dab3a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:36:57.717300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717266 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb744\" (UniqueName: \"kubernetes.io/projected/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-kube-api-access-zb744\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.717300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717294 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-trusted-ca-bundle\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.717300 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717303 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-config\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.717509 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717311 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-serving-cert\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.717509 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717319 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-console-oauth-config\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.717509 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717327 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-service-ca\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.717509 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.717335 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37673956-6dbc-422b-8f8c-5a3dd4dab3a3-oauth-serving-cert\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:36:57.825713 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.825676 2579 generic.go:356] "Generic (PLEG): container finished" podID="37673956-6dbc-422b-8f8c-5a3dd4dab3a3" containerID="6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740" exitCode=0 Jul 16 17:36:57.825864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.825727 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649ff4df44-w7thm" Jul 16 17:36:57.825864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.825731 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649ff4df44-w7thm" event={"ID":"37673956-6dbc-422b-8f8c-5a3dd4dab3a3","Type":"ContainerDied","Data":"6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740"} Jul 16 17:36:57.825864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.825779 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649ff4df44-w7thm" event={"ID":"37673956-6dbc-422b-8f8c-5a3dd4dab3a3","Type":"ContainerDied","Data":"fb775b5de6d805522f4d4c749f70109af955939eab9b28f4064009731022e4bb"} Jul 16 17:36:57.825864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.825820 2579 scope.go:117] "RemoveContainer" containerID="6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740" Jul 16 17:36:57.834589 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.834571 2579 scope.go:117] "RemoveContainer" containerID="6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740" Jul 16 17:36:57.834864 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:36:57.834842 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740\": container with ID starting with 6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740 not found: ID does not exist" containerID="6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740" Jul 16 17:36:57.834957 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.834869 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740"} err="failed to get container status \"6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740\": rpc error: code = NotFound desc = could not find container \"6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740\": container with ID starting with 6883dd6b594813a192c10940a79353cb0e392772460fa18130702ec96978f740 not found: ID does not exist" Jul 16 17:36:57.847773 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.847753 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-649ff4df44-w7thm"] Jul 16 17:36:57.851985 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:57.851966 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-649ff4df44-w7thm"] Jul 16 17:36:58.063270 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:36:58.063235 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37673956-6dbc-422b-8f8c-5a3dd4dab3a3" path="/var/lib/kubelet/pods/37673956-6dbc-422b-8f8c-5a3dd4dab3a3/volumes" Jul 16 17:37:22.677363 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:22.677323 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:22.697331 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:22.697304 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:22.937917 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:22.937836 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:24.926146 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:24.926114 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:33.947585 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:33.947548 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:34.054265 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:34.054237 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:35.336167 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:35.336126 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:35.742801 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:35.742771 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:40.741152 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741116 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:37:40.741573 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741395 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="prometheus" containerID="cri-o://8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" gracePeriod=600 Jul 16 17:37:40.741573 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741455 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-thanos" containerID="cri-o://58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" gracePeriod=600 Jul 16 17:37:40.741573 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741444 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="thanos-sidecar" containerID="cri-o://11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" gracePeriod=600 Jul 16 17:37:40.741573 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741501 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="config-reloader" containerID="cri-o://550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" gracePeriod=600 Jul 16 17:37:40.741573 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741432 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy" containerID="cri-o://ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" gracePeriod=600 Jul 16 17:37:40.741573 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.741485 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-web" containerID="cri-o://4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" gracePeriod=600 Jul 16 17:37:40.975905 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.975872 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:40.988610 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988584 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" exitCode=0 Jul 16 17:37:40.988610 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988606 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" exitCode=0 Jul 16 17:37:40.988610 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988614 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" exitCode=0 Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988619 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" exitCode=0 Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988625 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" exitCode=0 Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988629 2579 generic.go:356] "Generic (PLEG): container finished" podID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" exitCode=0 Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988656 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988714 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988761 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} Jul 16 17:37:40.988805 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988808 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} Jul 16 17:37:40.989173 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} Jul 16 17:37:40.989173 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988816 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:40.989173 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.988856 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e002fc7b-f4e1-4162-82d2-a0624be4ac06","Type":"ContainerDied","Data":"b89d092452d32f4c84e8dec0c1de1599726368bca4b42ec5a68eb5ff52ffc325"} Jul 16 17:37:40.996162 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:40.996146 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.004190 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.004129 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.012154 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.012137 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.020094 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.020074 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.027264 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.027250 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.034478 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.034462 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.042178 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.042159 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.042451 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.042434 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.042501 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.042459 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} err="failed to get container status \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" Jul 16 17:37:41.042501 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.042474 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.042739 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.042722 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.042780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.042743 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} err="failed to get container status \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" Jul 16 17:37:41.042780 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.042756 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.043009 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.042993 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.043122 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043015 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} err="failed to get container status \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" Jul 16 17:37:41.043122 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043069 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.043370 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.043355 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.043415 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043377 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} err="failed to get container status \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" Jul 16 17:37:41.043415 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043393 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.043604 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.043589 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.043642 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043608 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} err="failed to get container status \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" Jul 16 17:37:41.043642 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043619 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.043824 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.043810 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.043864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043827 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} err="failed to get container status \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" Jul 16 17:37:41.043864 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.043844 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.044044 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:37:41.044008 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.044092 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044043 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} err="failed to get container status \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" Jul 16 17:37:41.044092 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044058 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.044288 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044272 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} err="failed to get container status \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" Jul 16 17:37:41.044288 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044288 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.044468 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044451 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} err="failed to get container status \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" Jul 16 17:37:41.044510 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044468 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.044689 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044666 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} err="failed to get container status \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" Jul 16 17:37:41.044739 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044693 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.044923 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044904 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} err="failed to get container status \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" Jul 16 17:37:41.045004 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.044925 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.045162 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045141 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} err="failed to get container status \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" Jul 16 17:37:41.045230 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045164 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.045407 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045390 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} err="failed to get container status \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" Jul 16 17:37:41.045457 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045407 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.045616 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045600 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} err="failed to get container status \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" Jul 16 17:37:41.045662 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045617 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.045869 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045845 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} err="failed to get container status \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" Jul 16 17:37:41.045869 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.045865 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.046223 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046208 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} err="failed to get container status \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" Jul 16 17:37:41.046223 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046223 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.046434 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046417 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} err="failed to get container status \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" Jul 16 17:37:41.046503 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046436 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.046656 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046631 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} err="failed to get container status \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" Jul 16 17:37:41.046727 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046658 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.046892 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046875 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} err="failed to get container status \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" Jul 16 17:37:41.046938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.046893 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.047184 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047167 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} err="failed to get container status \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" Jul 16 17:37:41.047236 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047185 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.047503 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047441 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} err="failed to get container status \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" Jul 16 17:37:41.047503 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047503 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.047749 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047732 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} err="failed to get container status \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" Jul 16 17:37:41.047806 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047749 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.047956 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047939 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} err="failed to get container status \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" Jul 16 17:37:41.048012 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.047956 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.048196 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048174 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} err="failed to get container status \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" Jul 16 17:37:41.048262 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048197 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.048438 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048421 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} err="failed to get container status \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" Jul 16 17:37:41.048438 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048437 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.048640 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048619 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} err="failed to get container status \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" Jul 16 17:37:41.048715 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048641 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.048843 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048819 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} err="failed to get container status \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" Jul 16 17:37:41.048901 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.048845 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.049093 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049074 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} err="failed to get container status \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" Jul 16 17:37:41.049159 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049093 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.049340 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049322 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} err="failed to get container status \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" Jul 16 17:37:41.049392 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049341 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.049556 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049538 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} err="failed to get container status \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" Jul 16 17:37:41.049615 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049557 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.049762 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049741 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} err="failed to get container status \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" Jul 16 17:37:41.049804 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049763 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.049988 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049972 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} err="failed to get container status \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" Jul 16 17:37:41.049988 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.049988 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.050208 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050186 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} err="failed to get container status \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" Jul 16 17:37:41.050208 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050208 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.050425 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050407 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} err="failed to get container status \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" Jul 16 17:37:41.050495 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050427 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.050681 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050662 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} err="failed to get container status \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" Jul 16 17:37:41.050742 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050683 2579 scope.go:117] "RemoveContainer" containerID="58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2" Jul 16 17:37:41.050895 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050876 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2"} err="failed to get container status \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": rpc error: code = NotFound desc = could not find container \"58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2\": container with ID starting with 58c15fa6640e85ffd8552e4dd67c0e0a4e8e0f988a2a1b233b7a40a42843f5c2 not found: ID does not exist" Jul 16 17:37:41.050936 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.050897 2579 scope.go:117] "RemoveContainer" containerID="ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956" Jul 16 17:37:41.051131 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051114 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956"} err="failed to get container status \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": rpc error: code = NotFound desc = could not find container \"ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956\": container with ID starting with ecc6d93692de3f4c900fbac8bf9cdb0f68412ebff26331d817f1959d14646956 not found: ID does not exist" Jul 16 17:37:41.051210 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051131 2579 scope.go:117] "RemoveContainer" containerID="4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce" Jul 16 17:37:41.051337 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051314 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce"} err="failed to get container status \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": rpc error: code = NotFound desc = could not find container \"4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce\": container with ID starting with 4df90cf896a3803d3cf1328488f8786cb7321fc7f50fad6bb149f2c8303f9dce not found: ID does not exist" Jul 16 17:37:41.051337 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051339 2579 scope.go:117] "RemoveContainer" containerID="11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38" Jul 16 17:37:41.051556 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051540 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38"} err="failed to get container status \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": rpc error: code = NotFound desc = could not find container \"11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38\": container with ID starting with 11eee0e1c45f7fc533ff3963c3ae1d81908f300784e6e3cf64718cf2e3076a38 not found: ID does not exist" Jul 16 17:37:41.051601 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051558 2579 scope.go:117] "RemoveContainer" containerID="550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c" Jul 16 17:37:41.051785 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051766 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c"} err="failed to get container status \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": rpc error: code = NotFound desc = could not find container \"550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c\": container with ID starting with 550915ac86e433dbed9b78d4973af59f5984c10ef063b9ad2b8485c7c3b2b19c not found: ID does not exist" Jul 16 17:37:41.051849 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.051787 2579 scope.go:117] "RemoveContainer" containerID="8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2" Jul 16 17:37:41.052054 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.052011 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2"} err="failed to get container status \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": rpc error: code = NotFound desc = could not find container \"8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2\": container with ID starting with 8a48d446a82e41df4d1ffc68909805d16c8606210f68cc00de459553353051f2 not found: ID does not exist" Jul 16 17:37:41.052137 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.052062 2579 scope.go:117] "RemoveContainer" containerID="4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715" Jul 16 17:37:41.052285 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.052264 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715"} err="failed to get container status \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": rpc error: code = NotFound desc = could not find container \"4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715\": container with ID starting with 4655257db8142a620614a169ef317038e0e6b83a3c28c86d3b2479dfad1a1715 not found: ID does not exist" Jul 16 17:37:41.076851 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076829 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-db\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.076940 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076860 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-grpc-tls\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.076940 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076906 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-kube-rbac-proxy\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.076940 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076928 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-tls\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077127 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076955 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-metrics-client-ca\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077127 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076972 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config-out\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077127 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.076999 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077127 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077054 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077127 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077084 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-kubelet-serving-ca-bundle\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077127 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077110 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-tls-assets\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077134 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-web-config\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077168 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-2\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077214 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-1\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077249 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-thanos-prometheus-http-client-file\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077281 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-metrics-client-certs\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077305 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-serving-certs-ca-bundle\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077347 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8bgw\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-kube-api-access-w8bgw\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077406 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077387 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077426 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-0\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077455 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-trusted-ca-bundle\") pod \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\" (UID: \"e002fc7b-f4e1-4162-82d2-a0624be4ac06\") " Jul 16 17:37:41.077944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077594 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.077944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077716 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.077944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077885 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.077944 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.077904 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-metrics-client-ca\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.078268 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.078245 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.079236 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.078399 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:37:41.079236 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.079019 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.079390 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.079362 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.079644 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.079612 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.079926 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.079781 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.080320 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.080285 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.080574 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.080550 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:37:41.080731 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.080698 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.080731 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.080725 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:37:41.081216 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.081188 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.081653 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.081627 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config-out" (OuterVolumeSpecName: "config-out") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:37:41.081854 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.081828 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.082136 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.082110 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-kube-api-access-w8bgw" (OuterVolumeSpecName: "kube-api-access-w8bgw") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "kube-api-access-w8bgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:37:41.082216 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.082188 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.082260 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.082244 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config" (OuterVolumeSpecName: "config") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.082335 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.082319 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.092316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.092295 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-web-config" (OuterVolumeSpecName: "web-config") pod "e002fc7b-f4e1-4162-82d2-a0624be4ac06" (UID: "e002fc7b-f4e1-4162-82d2-a0624be4ac06"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:37:41.179078 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179015 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8bgw\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-kube-api-access-w8bgw\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179078 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179077 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179092 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179105 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179117 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-db\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179128 2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-grpc-tls\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179138 2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-kube-rbac-proxy\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179149 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179159 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config-out\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179170 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179182 2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-config\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179193 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179204 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e002fc7b-f4e1-4162-82d2-a0624be4ac06-tls-assets\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179215 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-web-config\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179226 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179238 2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179250 2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e002fc7b-f4e1-4162-82d2-a0624be4ac06-secret-metrics-client-certs\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.179316 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.179262 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e002fc7b-f4e1-4162-82d2-a0624be4ac06-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:37:41.313534 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.313497 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:37:41.318710 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.318644 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:37:41.349384 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349350 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:37:41.349678 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349661 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37673956-6dbc-422b-8f8c-5a3dd4dab3a3" containerName="console" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349681 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="37673956-6dbc-422b-8f8c-5a3dd4dab3a3" containerName="console" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349693 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349701 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349712 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="init-config-reloader" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349721 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="init-config-reloader" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349730 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="config-reloader" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349737 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="config-reloader" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349749 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-thanos" Jul 16 17:37:41.349755 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349757 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-thanos" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349768 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="thanos-sidecar" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349776 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="thanos-sidecar" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349790 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-web" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349798 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-web" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349811 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="prometheus" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349818 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="prometheus" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349878 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="prometheus" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349890 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="thanos-sidecar" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349900 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-web" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349908 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349916 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="37673956-6dbc-422b-8f8c-5a3dd4dab3a3" containerName="console" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349923 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="kube-rbac-proxy-thanos" Jul 16 17:37:41.350203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.349932 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" containerName="config-reloader" Jul 16 17:37:41.353873 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.353852 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.356180 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.356161 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 17:37:41.357305 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.357286 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 17:37:41.357669 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.357649 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 17:37:41.357810 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.357788 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 17:37:41.357934 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.357791 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-du89j81hkv2f4\"" Jul 16 17:37:41.357934 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.357867 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 17:37:41.358107 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358003 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 17:37:41.358107 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358052 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 17:37:41.358107 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358070 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 17:37:41.358107 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358056 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 17:37:41.358353 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358150 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 17:37:41.358353 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358264 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 17:37:41.358353 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358295 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 17:37:41.358353 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358305 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2kgrq\"" Jul 16 17:37:41.358576 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.358359 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 17:37:41.360282 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.360245 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 17:37:41.364455 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.364438 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 17:37:41.370812 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.370793 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:37:41.380645 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89711504-4ff1-49bd-984b-c3ad92830d00-config-out\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380749 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380653 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380749 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380684 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380749 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380865 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-config\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380865 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380865 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380865 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-web-config\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.380865 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380853 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380878 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380910 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89711504-4ff1-49bd-984b-c3ad92830d00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380954 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.380993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.381009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl8bz\" (UniqueName: \"kubernetes.io/projected/89711504-4ff1-49bd-984b-c3ad92830d00-kube-api-access-dl8bz\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.381038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381074 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.381059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381323 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.381104 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381323 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.381122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.381323 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.381149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481505 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481471 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89711504-4ff1-49bd-984b-c3ad92830d00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481505 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481506 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481579 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8bz\" (UniqueName: \"kubernetes.io/projected/89711504-4ff1-49bd-984b-c3ad92830d00-kube-api-access-dl8bz\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.481705 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481752 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89711504-4ff1-49bd-984b-c3ad92830d00-config-out\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481842 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-config\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482056 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.481921 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482606 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.482226 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.482743 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.482723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.483428 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.483463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-web-config\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.483497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.483440 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.483523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.484135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.484426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.484412 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485172 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.484443 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485172 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.484688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485172 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.484858 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/89711504-4ff1-49bd-984b-c3ad92830d00-config-out\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485172 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.484957 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.485350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485541 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.485525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.485988 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.485951 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/89711504-4ff1-49bd-984b-c3ad92830d00-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.486145 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.486019 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/89711504-4ff1-49bd-984b-c3ad92830d00-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.486848 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.486812 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.486938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.486854 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.487185 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.487166 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.487790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.487766 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-web-config\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.487933 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.487909 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-config\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.488248 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.488231 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/89711504-4ff1-49bd-984b-c3ad92830d00-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.490505 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.490487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl8bz\" (UniqueName: \"kubernetes.io/projected/89711504-4ff1-49bd-984b-c3ad92830d00-kube-api-access-dl8bz\") pod \"prometheus-k8s-0\" (UID: \"89711504-4ff1-49bd-984b-c3ad92830d00\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.664568 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.664494 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:41.804203 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.804172 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:37:41.808005 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:37:41.807980 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89711504_4ff1_49bd_984b_c3ad92830d00.slice/crio-9b9cbe45969e231f8adeb2f3a2c7ef8cf9e0e6dda79f148dab09c1f11ea4ba89 WatchSource:0}: Error finding container 9b9cbe45969e231f8adeb2f3a2c7ef8cf9e0e6dda79f148dab09c1f11ea4ba89: Status 404 returned error can't find the container with id 9b9cbe45969e231f8adeb2f3a2c7ef8cf9e0e6dda79f148dab09c1f11ea4ba89 Jul 16 17:37:41.994257 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.994230 2579 generic.go:356] "Generic (PLEG): container finished" podID="89711504-4ff1-49bd-984b-c3ad92830d00" containerID="0aea4ea3a7dbf0c6d1775db22d122a9d2a41f2f4197f386c935c9fbef9d0bd4b" exitCode=0 Jul 16 17:37:41.994422 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.994329 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerDied","Data":"0aea4ea3a7dbf0c6d1775db22d122a9d2a41f2f4197f386c935c9fbef9d0bd4b"} Jul 16 17:37:41.994422 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:41.994402 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"9b9cbe45969e231f8adeb2f3a2c7ef8cf9e0e6dda79f148dab09c1f11ea4ba89"} Jul 16 17:37:42.064486 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:42.064457 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e002fc7b-f4e1-4162-82d2-a0624be4ac06" path="/var/lib/kubelet/pods/e002fc7b-f4e1-4162-82d2-a0624be4ac06/volumes" Jul 16 17:37:43.002731 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.002689 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"43b9d5481ce983f1a6db60f36caf4f797c48c2d67e7b7b35a6d2ba5c40a59c0f"} Jul 16 17:37:43.003109 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.002757 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"6971adabfdb2e48f0e4e6e263407e12ba9fd9c1ed56aa66db4e2721c74b38905"} Jul 16 17:37:43.003109 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.002784 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"ea952676641ed462511f1fb37f0b589074696c077d258ca21f7d9b06a89d98eb"} Jul 16 17:37:43.003109 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.002807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"89d7f8a0ac8128b74111ea891d29de90032dc1fac2848c91607cccae6201f772"} Jul 16 17:37:43.003109 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.002830 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"819169330df54cf5e01cd774b330d6dd25ad420038aefd21ddbad9f4a2b51221"} Jul 16 17:37:43.003109 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.002854 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"89711504-4ff1-49bd-984b-c3ad92830d00","Type":"ContainerStarted","Data":"99a9311a0bce3efdedd532dd2ec39035b33201cfc5fa58ad16869dc29477bd6c"} Jul 16 17:37:43.043060 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:43.042914 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.02448267 podStartE2EDuration="2.042892312s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.409642ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:37:43.041122938 +0000 UTC m=+245.527415949" watchObservedRunningTime="2026-07-16 17:37:43.042892312 +0000 UTC m=+245.529185318" Jul 16 17:37:44.581945 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:44.581916 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:44.642239 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:44.642208 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:46.665075 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:46.665021 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:37:55.735419 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:55.735378 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:55.793732 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:55.793693 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:37:55.850361 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:37:55.850323 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:38:37.937514 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:37.937488 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:38:37.948451 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:37.948308 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 17:38:37.950828 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:37.950803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:38:38.368043 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:38.367986 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:38:41.665464 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:41.665426 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:38:41.682197 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:41.682166 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:38:42.232419 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:42.232389 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:38:50.627322 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:50.627261 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:38:50.687288 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:50.687257 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:38:56.531625 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:56.531588 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:38:56.601557 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:38:56.601528 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:02.528150 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:02.528114 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:02.587656 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:02.587615 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:03.040789 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:03.040756 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:07.021013 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:07.020969 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:07.079907 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:07.079860 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:07.142642 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:07.140804 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:16.690516 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:16.690468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:16.792206 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:39:16.792169 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:40:35.939656 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:40:35.939602 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:40:37.795156 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:40:37.795112 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:03.720219 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:03.720181 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:03.827960 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:03.827924 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:09.826688 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:09.826643 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.106906 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.106815 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.316617 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.316583 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.405063 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.404958 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.584546 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.584508 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.705767 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.705656 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.789781 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.789747 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:10.853445 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:10.853412 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:11.250512 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:11.250471 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:11.445666 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:11.445634 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:11.544052 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:11.543947 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:11.649635 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:11.649601 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:34.840144 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:34.840110 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:47.371751 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:47.371717 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:47.445482 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:47.445445 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:49.225596 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:49.225560 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:41:49.327655 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:41:49.327627 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:17.495867 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:17.495827 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:17.573515 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:17.573483 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:17.672018 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:17.671974 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:17.773227 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:17.773135 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:18.554035 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:18.554000 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:18.691520 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:18.691491 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:51.327940 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:51.327855 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:42:51.389151 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:42:51.389113 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:02.025288 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:02.025253 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:02.127575 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:02.127543 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:02.186084 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:02.186051 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:32.478655 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:32.478592 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:32.564404 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:32.564368 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:32.669073 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:32.669020 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:33.006806 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:33.006768 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:33.335863 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:33.335769 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:33.419592 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:33.419558 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:37.914723 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:37.914684 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:43:37.968483 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:37.968457 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:43:37.972742 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:37.972720 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:43:47.890013 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.889982 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-tglc2"] Jul 16 17:43:47.898053 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.897987 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tglc2" Jul 16 17:43:47.900517 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.900464 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 16 17:43:47.900796 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.900774 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tglc2"] Jul 16 17:43:47.900889 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.900779 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 16 17:43:47.900889 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.900848 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 16 17:43:47.901567 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:47.901551 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xvd56\"" Jul 16 17:43:48.033643 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.033610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vwxv\" (UniqueName: \"kubernetes.io/projected/2df27344-a459-4c33-89ce-82626795f04d-kube-api-access-9vwxv\") pod \"s3-init-tglc2\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " pod="kserve/s3-init-tglc2" Jul 16 17:43:48.033828 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.033668 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2df27344-a459-4c33-89ce-82626795f04d-model-cache\") pod \"s3-init-tglc2\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " pod="kserve/s3-init-tglc2" Jul 16 17:43:48.135008 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.134971 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwxv\" (UniqueName: \"kubernetes.io/projected/2df27344-a459-4c33-89ce-82626795f04d-kube-api-access-9vwxv\") pod \"s3-init-tglc2\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " pod="kserve/s3-init-tglc2" Jul 16 17:43:48.135274 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.135070 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2df27344-a459-4c33-89ce-82626795f04d-model-cache\") pod \"s3-init-tglc2\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " pod="kserve/s3-init-tglc2" Jul 16 17:43:48.135393 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.135377 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2df27344-a459-4c33-89ce-82626795f04d-model-cache\") pod \"s3-init-tglc2\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " pod="kserve/s3-init-tglc2" Jul 16 17:43:48.145280 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.145217 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vwxv\" (UniqueName: \"kubernetes.io/projected/2df27344-a459-4c33-89ce-82626795f04d-kube-api-access-9vwxv\") pod \"s3-init-tglc2\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " pod="kserve/s3-init-tglc2" Jul 16 17:43:48.207715 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.207676 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tglc2" Jul 16 17:43:48.333933 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.333903 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tglc2"] Jul 16 17:43:48.337283 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:43:48.337245 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2df27344_a459_4c33_89ce_82626795f04d.slice/crio-26271dfbc3d88c4dd4345c11c185beba7d8fa87d7e83cec7ae1d6de18b5bcea3 WatchSource:0}: Error finding container 26271dfbc3d88c4dd4345c11c185beba7d8fa87d7e83cec7ae1d6de18b5bcea3: Status 404 returned error can't find the container with id 26271dfbc3d88c4dd4345c11c185beba7d8fa87d7e83cec7ae1d6de18b5bcea3 Jul 16 17:43:48.339146 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.339126 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:43:48.350820 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:48.350796 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tglc2" event={"ID":"2df27344-a459-4c33-89ce-82626795f04d","Type":"ContainerStarted","Data":"26271dfbc3d88c4dd4345c11c185beba7d8fa87d7e83cec7ae1d6de18b5bcea3"} Jul 16 17:43:52.371679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:52.371641 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tglc2" event={"ID":"2df27344-a459-4c33-89ce-82626795f04d","Type":"ContainerStarted","Data":"ca28c49261b546346a4c8d55aec6f6960e31a635960291910e98a002bcfab6e7"} Jul 16 17:43:53.377009 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:53.376923 2579 generic.go:356] "Generic (PLEG): container finished" podID="2df27344-a459-4c33-89ce-82626795f04d" containerID="ca28c49261b546346a4c8d55aec6f6960e31a635960291910e98a002bcfab6e7" exitCode=0 Jul 16 17:43:53.377009 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:53.377005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tglc2" event={"ID":"2df27344-a459-4c33-89ce-82626795f04d","Type":"ContainerDied","Data":"ca28c49261b546346a4c8d55aec6f6960e31a635960291910e98a002bcfab6e7"} Jul 16 17:43:58.402044 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:58.401998 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tglc2" event={"ID":"2df27344-a459-4c33-89ce-82626795f04d","Type":"ContainerStarted","Data":"bd099db1a4896db4e24f2b2e3886e8c2ddf93149bb66c8ea1caab1d7f63c21de"} Jul 16 17:43:58.421503 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:43:58.421445 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-tglc2" podStartSLOduration=2.5270202729999998 podStartE2EDuration="11.421428121s" totalImagesPullingTime="7.389491704s" totalInitContainerRuntime="1.504916144s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:43:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:43:58.420095709 +0000 UTC m=+620.906388713" watchObservedRunningTime="2026-07-16 17:43:58.421428121 +0000 UTC m=+620.907721125" Jul 16 17:44:38.548646 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:44:38.548609 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:38.610394 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:44:38.610357 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:56.472015 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:44:56.471981 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:02.631262 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:02.631230 2579 generic.go:356] "Generic (PLEG): container finished" podID="2df27344-a459-4c33-89ce-82626795f04d" containerID="bd099db1a4896db4e24f2b2e3886e8c2ddf93149bb66c8ea1caab1d7f63c21de" exitCode=0 Jul 16 17:45:02.631632 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:02.631304 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tglc2" event={"ID":"2df27344-a459-4c33-89ce-82626795f04d","Type":"ContainerDied","Data":"bd099db1a4896db4e24f2b2e3886e8c2ddf93149bb66c8ea1caab1d7f63c21de"} Jul 16 17:45:03.753214 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.753193 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tglc2" Jul 16 17:45:03.859270 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.859235 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vwxv\" (UniqueName: \"kubernetes.io/projected/2df27344-a459-4c33-89ce-82626795f04d-kube-api-access-9vwxv\") pod \"2df27344-a459-4c33-89ce-82626795f04d\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " Jul 16 17:45:03.859438 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.859285 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2df27344-a459-4c33-89ce-82626795f04d-model-cache\") pod \"2df27344-a459-4c33-89ce-82626795f04d\" (UID: \"2df27344-a459-4c33-89ce-82626795f04d\") " Jul 16 17:45:03.861582 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.861549 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df27344-a459-4c33-89ce-82626795f04d-kube-api-access-9vwxv" (OuterVolumeSpecName: "kube-api-access-9vwxv") pod "2df27344-a459-4c33-89ce-82626795f04d" (UID: "2df27344-a459-4c33-89ce-82626795f04d"). InnerVolumeSpecName "kube-api-access-9vwxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:45:03.877290 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.877266 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df27344-a459-4c33-89ce-82626795f04d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2df27344-a459-4c33-89ce-82626795f04d" (UID: "2df27344-a459-4c33-89ce-82626795f04d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:45:03.960586 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.960493 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vwxv\" (UniqueName: \"kubernetes.io/projected/2df27344-a459-4c33-89ce-82626795f04d-kube-api-access-9vwxv\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:45:03.960586 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:03.960519 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2df27344-a459-4c33-89ce-82626795f04d-model-cache\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:45:04.639740 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:04.639706 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tglc2" Jul 16 17:45:04.639740 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:04.639718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tglc2" event={"ID":"2df27344-a459-4c33-89ce-82626795f04d","Type":"ContainerDied","Data":"26271dfbc3d88c4dd4345c11c185beba7d8fa87d7e83cec7ae1d6de18b5bcea3"} Jul 16 17:45:04.639941 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:04.639753 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26271dfbc3d88c4dd4345c11c185beba7d8fa87d7e83cec7ae1d6de18b5bcea3" Jul 16 17:45:05.003349 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:05.003319 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:05.104452 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:05.104418 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:15.162630 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:15.162597 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:16.420963 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.420930 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:16.658657 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.658622 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:16.758599 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.758550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:16.762522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.762494 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8"] Jul 16 17:45:16.762842 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.762826 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df27344-a459-4c33-89ce-82626795f04d" containerName="s3-init" Jul 16 17:45:16.762916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.762843 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df27344-a459-4c33-89ce-82626795f04d" containerName="s3-init" Jul 16 17:45:16.762916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.762871 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df27344-a459-4c33-89ce-82626795f04d" containerName="download-hf-model" Jul 16 17:45:16.762916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.762881 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df27344-a459-4c33-89ce-82626795f04d" containerName="download-hf-model" Jul 16 17:45:16.763098 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.762952 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df27344-a459-4c33-89ce-82626795f04d" containerName="s3-init" Jul 16 17:45:16.765980 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.765961 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.768421 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.768389 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wv2xq\"" Jul 16 17:45:16.768527 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.768503 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:45:16.768815 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.768796 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:45:16.768815 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.768808 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 16 17:45:16.768984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.768798 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 16 17:45:16.768984 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.768820 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 16 17:45:16.769533 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.769513 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 16 17:45:16.778565 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.778545 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8"] Jul 16 17:45:16.826610 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.826574 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:16.865387 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.865361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d10dbe18-5da2-481f-9579-f985ccb3e492-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.865538 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.865396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d10dbe18-5da2-481f-9579-f985ccb3e492-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.865538 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.865421 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mnmk\" (UniqueName: \"kubernetes.io/projected/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-api-access-2mnmk\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.865538 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.865459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.865538 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.865496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.912400 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.912345 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:16.966679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.966399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d10dbe18-5da2-481f-9579-f985ccb3e492-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.966679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.966450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d10dbe18-5da2-481f-9579-f985ccb3e492-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.966679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.966490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mnmk\" (UniqueName: \"kubernetes.io/projected/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-api-access-2mnmk\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.966679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.966515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.966679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.966548 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.967467 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.967316 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.967467 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.967340 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d10dbe18-5da2-481f-9579-f985ccb3e492-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.967467 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.967410 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.968212 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.968148 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg"] Jul 16 17:45:16.969983 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.969953 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d10dbe18-5da2-481f-9579-f985ccb3e492-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.971685 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.971666 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:16.974066 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.974042 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 16 17:45:16.974165 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.974079 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 16 17:45:16.980978 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.980954 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg"] Jul 16 17:45:16.982790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.982770 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mnmk\" (UniqueName: \"kubernetes.io/projected/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-api-access-2mnmk\") pod \"isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:16.984075 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:16.984052 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:17.061916 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.061835 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:17.068059 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.068020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.068175 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.068094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.068175 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.068126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.068175 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.068170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.068329 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.068204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cznc2\" (UniqueName: \"kubernetes.io/projected/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-api-access-cznc2\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.076790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.076767 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:17.169170 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.169135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.169348 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.169201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.169348 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.169245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cznc2\" (UniqueName: \"kubernetes.io/projected/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-api-access-cznc2\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.169348 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.169311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.169491 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:45:17.169342 2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 16 17:45:17.169491 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:45:17.169435 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls podName:fbf50dfb-21f7-4f35-8496-a8cfaa013804 nodeName:}" failed. No retries permitted until 2026-07-16 17:45:17.669413445 +0000 UTC m=+700.155706428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-wvdwg" (UID: "fbf50dfb-21f7-4f35-8496-a8cfaa013804") : secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 16 17:45:17.169724 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.169706 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.170000 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.169979 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.170102 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.170018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.170583 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.170561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.188679 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.188623 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cznc2\" (UniqueName: \"kubernetes.io/projected/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-api-access-cznc2\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.255556 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.255533 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8"] Jul 16 17:45:17.258355 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:45:17.258321 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10dbe18_5da2_481f_9579_f985ccb3e492.slice/crio-b0132e2a9c57a10dd631adfb958e2237753a140a6b9199ac395c2f2847a82463 WatchSource:0}: Error finding container b0132e2a9c57a10dd631adfb958e2237753a140a6b9199ac395c2f2847a82463: Status 404 returned error can't find the container with id b0132e2a9c57a10dd631adfb958e2237753a140a6b9199ac395c2f2847a82463 Jul 16 17:45:17.330860 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.330831 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:17.429935 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.429899 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:17.674016 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.673928 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.676426 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.676406 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-wvdwg\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:17.689757 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.689730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerStarted","Data":"ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d"} Jul 16 17:45:17.689876 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.689781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerStarted","Data":"b0132e2a9c57a10dd631adfb958e2237753a140a6b9199ac395c2f2847a82463"} Jul 16 17:45:17.899119 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:17.899083 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:18.027415 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:18.027388 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg"] Jul 16 17:45:18.030668 ip-10-0-141-151 kubenswrapper[2579]: W0716 17:45:18.030645 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf50dfb_21f7_4f35_8496_a8cfaa013804.slice/crio-bb2436583eed8c5d36730d93ae4d400971b4f69d3a8830f6749340eb5c9ee947 WatchSource:0}: Error finding container bb2436583eed8c5d36730d93ae4d400971b4f69d3a8830f6749340eb5c9ee947: Status 404 returned error can't find the container with id bb2436583eed8c5d36730d93ae4d400971b4f69d3a8830f6749340eb5c9ee947 Jul 16 17:45:18.695197 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:18.695163 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerStarted","Data":"4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423"} Jul 16 17:45:18.695583 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:18.695255 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerStarted","Data":"bb2436583eed8c5d36730d93ae4d400971b4f69d3a8830f6749340eb5c9ee947"} Jul 16 17:45:21.709181 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:21.709150 2579 generic.go:356] "Generic (PLEG): container finished" podID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerID="ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d" exitCode=0 Jul 16 17:45:21.709181 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:21.709186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerDied","Data":"ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d"} Jul 16 17:45:22.720849 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:22.720809 2579 generic.go:356] "Generic (PLEG): container finished" podID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerID="4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423" exitCode=0 Jul 16 17:45:22.721328 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:22.720864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerDied","Data":"4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423"} Jul 16 17:45:37.799187 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:37.799144 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerStarted","Data":"8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71"} Jul 16 17:45:43.122853 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:43.122812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:48.846999 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.846956 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerStarted","Data":"fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301"} Jul 16 17:45:48.847476 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.847188 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:48.847476 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.847240 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:48.848857 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.848813 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:45:48.849446 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.849429 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerStarted","Data":"eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8"} Jul 16 17:45:48.849529 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.849467 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerStarted","Data":"cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5"} Jul 16 17:45:48.849602 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.849587 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:48.867155 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.867114 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podStartSLOduration=2.9537156209999997 podStartE2EDuration="32.86710035s" totalImagesPullingTime="26.058812531s" totalInitContainerRuntime="3.854572198s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:45:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:45:48.866138601 +0000 UTC m=+731.352431605" watchObservedRunningTime="2026-07-16 17:45:48.86710035 +0000 UTC m=+731.353393354" Jul 16 17:45:48.885854 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:48.885809 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podStartSLOduration=3.610263539 podStartE2EDuration="32.885795611s" totalImagesPullingTime="25.171068355s" totalInitContainerRuntime="4.104463717s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:45:48.884107434 +0000 UTC m=+731.370400440" watchObservedRunningTime="2026-07-16 17:45:48.885795611 +0000 UTC m=+731.372088677" Jul 16 17:45:49.853511 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:49.853479 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:49.853938 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:49.853527 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:45:49.854719 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:49.854696 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:45:50.856678 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:50.856637 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:45:54.858064 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:54.858010 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:45:54.858534 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:54.858510 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:45:55.861110 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:55.861083 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:45:55.861600 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:45:55.861570 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:46:04.859234 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:04.859196 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:46:05.862067 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:05.862012 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:46:14.858990 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:14.858930 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:46:15.861794 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:15.861755 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:46:24.858518 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:24.858478 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:46:25.861541 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:25.861499 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:46:34.859157 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:34.859114 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:46:35.862324 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:35.862285 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jul 16 17:46:36.494308 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:36.494271 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:36.554294 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:36.554256 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:44.858939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:44.858899 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused" Jul 16 17:46:45.863071 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:45.863039 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:46:46.610799 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:46.610758 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:46.688329 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:46.688296 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:46.766274 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:46.766238 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:46.861181 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:46.861092 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:46.960561 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:46.960517 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:47.063981 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:47.063948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:47.285680 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:47.285653 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:54.126197 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:54.126152 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:54.180326 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:54.180298 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:46:54.859076 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:46:54.859049 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:47:16.541002 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:16.540964 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:16.715930 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:16.715881 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:26.671189 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.671151 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:26.809061 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.809012 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8"] Jul 16 17:47:26.809359 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.809331 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" containerID="cri-o://8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71" gracePeriod=30 Jul 16 17:47:26.809496 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.809436 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kube-rbac-proxy" containerID="cri-o://fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301" gracePeriod=30 Jul 16 17:47:26.817092 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.817064 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:26.861671 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.861641 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg"] Jul 16 17:47:26.861908 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.861881 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" containerID="cri-o://cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5" gracePeriod=30 Jul 16 17:47:26.861962 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.861908 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kube-rbac-proxy" containerID="cri-o://eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8" gracePeriod=30 Jul 16 17:47:26.895674 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.895645 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:26.966650 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:26.966612 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:27.038107 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.038069 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:27.139793 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.139753 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:27.224199 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.224108 2579 generic.go:356] "Generic (PLEG): container finished" podID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerID="fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301" exitCode=2 Jul 16 17:47:27.224351 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.224244 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerDied","Data":"fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301"} Jul 16 17:47:27.229066 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.229035 2579 generic.go:356] "Generic (PLEG): container finished" podID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerID="eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8" exitCode=2 Jul 16 17:47:27.229066 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.229059 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerDied","Data":"eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8"} Jul 16 17:47:27.668722 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:27.668626 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:29.853769 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:29.853721 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.22:8643/healthz\": dial tcp 10.132.0.22:8643: connect: connection refused" Jul 16 17:47:30.508181 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.508159 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:47:30.656893 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.656803 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kserve-provision-location\") pod \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " Jul 16 17:47:30.656893 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.656860 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls\") pod \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " Jul 16 17:47:30.656893 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.656883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-cabundle-cert\") pod \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " Jul 16 17:47:30.657198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.656914 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cznc2\" (UniqueName: \"kubernetes.io/projected/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-api-access-cznc2\") pod \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " Jul 16 17:47:30.657198 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.656979 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-rbac-proxy-sar-config\") pod \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\" (UID: \"fbf50dfb-21f7-4f35-8496-a8cfaa013804\") " Jul 16 17:47:30.657283 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.657208 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "fbf50dfb-21f7-4f35-8496-a8cfaa013804" (UID: "fbf50dfb-21f7-4f35-8496-a8cfaa013804"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:47:30.657333 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.657287 2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kserve-provision-location\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:30.657396 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.657368 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "fbf50dfb-21f7-4f35-8496-a8cfaa013804" (UID: "fbf50dfb-21f7-4f35-8496-a8cfaa013804"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:47:30.657448 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.657422 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fbf50dfb-21f7-4f35-8496-a8cfaa013804" (UID: "fbf50dfb-21f7-4f35-8496-a8cfaa013804"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:47:30.659219 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.659196 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-api-access-cznc2" (OuterVolumeSpecName: "kube-api-access-cznc2") pod "fbf50dfb-21f7-4f35-8496-a8cfaa013804" (UID: "fbf50dfb-21f7-4f35-8496-a8cfaa013804"). InnerVolumeSpecName "kube-api-access-cznc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:47:30.659288 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.659253 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fbf50dfb-21f7-4f35-8496-a8cfaa013804" (UID: "fbf50dfb-21f7-4f35-8496-a8cfaa013804"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:47:30.758680 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.758644 2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:30.758680 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.758672 2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbf50dfb-21f7-4f35-8496-a8cfaa013804-proxy-tls\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:30.758680 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.758684 2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbf50dfb-21f7-4f35-8496-a8cfaa013804-cabundle-cert\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:30.758939 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:30.758695 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cznc2\" (UniqueName: \"kubernetes.io/projected/fbf50dfb-21f7-4f35-8496-a8cfaa013804-kube-api-access-cznc2\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:31.245112 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.245080 2579 generic.go:356] "Generic (PLEG): container finished" podID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerID="cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5" exitCode=0 Jul 16 17:47:31.245478 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.245139 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" Jul 16 17:47:31.245478 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.245174 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerDied","Data":"cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5"} Jul 16 17:47:31.245478 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.245245 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg" event={"ID":"fbf50dfb-21f7-4f35-8496-a8cfaa013804","Type":"ContainerDied","Data":"bb2436583eed8c5d36730d93ae4d400971b4f69d3a8830f6749340eb5c9ee947"} Jul 16 17:47:31.245478 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.245285 2579 scope.go:117] "RemoveContainer" containerID="eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8" Jul 16 17:47:31.253790 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.253774 2579 scope.go:117] "RemoveContainer" containerID="cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5" Jul 16 17:47:31.261304 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.261284 2579 scope.go:117] "RemoveContainer" containerID="4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423" Jul 16 17:47:31.269053 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269015 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg"] Jul 16 17:47:31.269214 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269191 2579 scope.go:117] "RemoveContainer" containerID="eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8" Jul 16 17:47:31.269489 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:47:31.269473 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8\": container with ID starting with eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8 not found: ID does not exist" containerID="eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8" Jul 16 17:47:31.269533 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269498 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8"} err="failed to get container status \"eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8\": rpc error: code = NotFound desc = could not find container \"eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8\": container with ID starting with eef230cbcdfc9875b4cbea7d27acb4d534c6bdcdacce30f115ccd55505045ad8 not found: ID does not exist" Jul 16 17:47:31.269533 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269516 2579 scope.go:117] "RemoveContainer" containerID="cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5" Jul 16 17:47:31.269732 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:47:31.269718 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5\": container with ID starting with cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5 not found: ID does not exist" containerID="cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5" Jul 16 17:47:31.269773 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269733 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5"} err="failed to get container status \"cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5\": rpc error: code = NotFound desc = could not find container \"cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5\": container with ID starting with cd3538065f5c582773fb26dbdf8cde8c94ab40ea0e511b9f995dae252d70a5e5 not found: ID does not exist" Jul 16 17:47:31.269773 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269745 2579 scope.go:117] "RemoveContainer" containerID="4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423" Jul 16 17:47:31.269943 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:47:31.269928 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423\": container with ID starting with 4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423 not found: ID does not exist" containerID="4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423" Jul 16 17:47:31.269982 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.269947 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423"} err="failed to get container status \"4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423\": rpc error: code = NotFound desc = could not find container \"4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423\": container with ID starting with 4a31edbff8ee45c1a3f5c23068f9a8f3af343b696a77734f7bf55cfd557a0423 not found: ID does not exist" Jul 16 17:47:31.272655 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.272636 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-wvdwg"] Jul 16 17:47:31.461666 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.461645 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:47:31.565313 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565214 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-cabundle-cert\") pod \"d10dbe18-5da2-481f-9579-f985ccb3e492\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " Jul 16 17:47:31.565313 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565283 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mnmk\" (UniqueName: \"kubernetes.io/projected/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-api-access-2mnmk\") pod \"d10dbe18-5da2-481f-9579-f985ccb3e492\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " Jul 16 17:47:31.565546 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565385 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-rbac-proxy-sar-config\") pod \"d10dbe18-5da2-481f-9579-f985ccb3e492\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " Jul 16 17:47:31.565546 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565440 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d10dbe18-5da2-481f-9579-f985ccb3e492-proxy-tls\") pod \"d10dbe18-5da2-481f-9579-f985ccb3e492\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " Jul 16 17:47:31.565546 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565485 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d10dbe18-5da2-481f-9579-f985ccb3e492-kserve-provision-location\") pod \"d10dbe18-5da2-481f-9579-f985ccb3e492\" (UID: \"d10dbe18-5da2-481f-9579-f985ccb3e492\") " Jul 16 17:47:31.565692 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565659 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d10dbe18-5da2-481f-9579-f985ccb3e492" (UID: "d10dbe18-5da2-481f-9579-f985ccb3e492"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:47:31.565737 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565712 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d10dbe18-5da2-481f-9579-f985ccb3e492" (UID: "d10dbe18-5da2-481f-9579-f985ccb3e492"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:47:31.565917 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.565895 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d10dbe18-5da2-481f-9579-f985ccb3e492-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d10dbe18-5da2-481f-9579-f985ccb3e492" (UID: "d10dbe18-5da2-481f-9579-f985ccb3e492"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:47:31.567596 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.567574 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-api-access-2mnmk" (OuterVolumeSpecName: "kube-api-access-2mnmk") pod "d10dbe18-5da2-481f-9579-f985ccb3e492" (UID: "d10dbe18-5da2-481f-9579-f985ccb3e492"). InnerVolumeSpecName "kube-api-access-2mnmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:47:31.567652 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.567615 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d10dbe18-5da2-481f-9579-f985ccb3e492-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d10dbe18-5da2-481f-9579-f985ccb3e492" (UID: "d10dbe18-5da2-481f-9579-f985ccb3e492"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:47:31.666499 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.666466 2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-rbac-proxy-sar-config\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:31.666499 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.666493 2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d10dbe18-5da2-481f-9579-f985ccb3e492-proxy-tls\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:31.666499 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.666506 2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d10dbe18-5da2-481f-9579-f985ccb3e492-kserve-provision-location\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:31.666720 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.666519 2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d10dbe18-5da2-481f-9579-f985ccb3e492-cabundle-cert\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:31.666720 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:31.666530 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mnmk\" (UniqueName: \"kubernetes.io/projected/d10dbe18-5da2-481f-9579-f985ccb3e492-kube-api-access-2mnmk\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 17:47:32.067627 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.067593 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" path="/var/lib/kubelet/pods/fbf50dfb-21f7-4f35-8496-a8cfaa013804/volumes" Jul 16 17:47:32.251769 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.251730 2579 generic.go:356] "Generic (PLEG): container finished" podID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerID="8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71" exitCode=0 Jul 16 17:47:32.252239 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.251808 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" Jul 16 17:47:32.252239 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.251816 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerDied","Data":"8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71"} Jul 16 17:47:32.252239 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.251889 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8" event={"ID":"d10dbe18-5da2-481f-9579-f985ccb3e492","Type":"ContainerDied","Data":"b0132e2a9c57a10dd631adfb958e2237753a140a6b9199ac395c2f2847a82463"} Jul 16 17:47:32.252239 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.251925 2579 scope.go:117] "RemoveContainer" containerID="fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301" Jul 16 17:47:32.260489 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.260472 2579 scope.go:117] "RemoveContainer" containerID="8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71" Jul 16 17:47:32.267814 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.267795 2579 scope.go:117] "RemoveContainer" containerID="ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d" Jul 16 17:47:32.272078 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.272052 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8"] Jul 16 17:47:32.275684 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.275666 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-d779bb8df-bgvc8"] Jul 16 17:47:32.275743 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.275689 2579 scope.go:117] "RemoveContainer" containerID="fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301" Jul 16 17:47:32.275975 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:47:32.275955 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301\": container with ID starting with fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301 not found: ID does not exist" containerID="fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301" Jul 16 17:47:32.276136 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.275982 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301"} err="failed to get container status \"fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301\": rpc error: code = NotFound desc = could not find container \"fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301\": container with ID starting with fcc47433b0cdd1f6801e777ba47c3aff107a3839168d70ecbc52cc9a02fbe301 not found: ID does not exist" Jul 16 17:47:32.276136 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.276001 2579 scope.go:117] "RemoveContainer" containerID="8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71" Jul 16 17:47:32.276264 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:47:32.276249 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71\": container with ID starting with 8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71 not found: ID does not exist" containerID="8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71" Jul 16 17:47:32.276302 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.276268 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71"} err="failed to get container status \"8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71\": rpc error: code = NotFound desc = could not find container \"8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71\": container with ID starting with 8b12b1e4dda2f4ea3c60e1504361c378d555070b3d62fa4d1b9a11944cd01f71 not found: ID does not exist" Jul 16 17:47:32.276302 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.276280 2579 scope.go:117] "RemoveContainer" containerID="ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d" Jul 16 17:47:32.276480 ip-10-0-141-151 kubenswrapper[2579]: E0716 17:47:32.276463 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d\": container with ID starting with ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d not found: ID does not exist" containerID="ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d" Jul 16 17:47:32.276522 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:32.276485 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d"} err="failed to get container status \"ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d\": rpc error: code = NotFound desc = could not find container \"ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d\": container with ID starting with ed6995e61c17227de1f2ac7cf2247c908a9589d21bc19232c185264253b70e9d not found: ID does not exist" Jul 16 17:47:34.064104 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:34.064071 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" path="/var/lib/kubelet/pods/d10dbe18-5da2-481f-9579-f985ccb3e492/volumes" Jul 16 17:47:46.839757 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:46.839715 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:47:46.943470 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:47:46.943432 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:48:26.865726 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:48:26.865690 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:48:27.137116 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:48:27.137012 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:48:37.993227 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:48:37.993142 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:48:37.995270 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:48:37.995249 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:50:25.231842 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:50:25.231802 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:50:25.340381 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:50:25.340347 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:51:41.580297 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:51:41.580211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:51:46.578078 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:51:46.578016 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:53:38.020786 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:53:38.020749 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:53:38.021407 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:53:38.020853 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:54:34.551673 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:54:34.551629 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:01.615462 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:01.615371 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:01.776122 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:01.776089 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:01.866011 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:01.865928 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:01.939421 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:01.939377 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:02.009865 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:02.009831 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:02.126047 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:02.125944 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:02.236476 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:02.236447 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:41.478437 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:41.478383 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:41.571823 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:41.571780 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:41.657558 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:41.657520 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:41.727941 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:41.727901 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:41.810281 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:41.810193 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:42.007493 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:42.007460 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:56:42.586405 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:56:42.586374 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:01.878366 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:01.878333 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:02.977146 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:02.977109 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:12.076917 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:12.076878 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:12.190642 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:12.190596 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:12.260609 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:12.260550 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:12.338644 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:12.338538 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:12.408079 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:12.408042 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:12.510832 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:12.510798 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:41.697557 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:41.697463 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:42.104844 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:42.104812 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:51.794157 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:51.794126 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:51.875658 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:51.875623 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:51.956459 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:51.956430 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:52.042042 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:52.041994 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:52.112210 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:52.112129 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:52.168572 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:57:52.168537 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:58:12.394339 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:12.394301 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:58:12.720504 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:12.720471 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:58:33.698332 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:33.698297 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:58:38.051037 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:38.050994 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:58:38.051731 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:38.051714 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 17:58:52.013715 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:52.013668 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 17:58:52.120133 ip-10-0-141-151 kubenswrapper[2579]: I0716 17:58:52.120070 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:03:38.076490 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:03:38.076464 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:03:38.077903 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:03:38.077880 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:04:30.304769 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:04:30.304738 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:27.023749 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:27.023709 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:27.130377 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:27.130337 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:27.210050 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:27.210001 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:27.343929 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:27.343834 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:27.602595 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:27.602510 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:28.004368 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:28.004329 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:06:41.820885 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:06:41.820849 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:06.579040 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:06.578999 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:06.688255 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:06.688211 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:06.759125 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:06.759093 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:06.861817 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:06.861727 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:06.969186 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:06.969155 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:07.229955 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:07.229912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:27.245157 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:27.245005 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:27.480096 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:27.480060 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:37.347045 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:37.346991 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:37.451069 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:37.451035 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:37.522948 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:37.522912 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:37.633841 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:37.633720 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:37.780225 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:37.780192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:37.879112 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:37.879074 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:39.831620 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:07:39.831581 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:08:06.802525 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:08:06.802479 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:08:07.070323 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:08:07.070235 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:08:37.605111 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:08:37.605075 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:08:37.831072 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:08:37.831019 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:08:38.100368 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:08:38.100340 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:08:38.102083 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:08:38.102062 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:11:49.048280 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:11:49.048244 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:12:57.600330 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:12:57.600253 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:13:22.385963 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:13:22.385928 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:13:38.123101 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:13:38.123072 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:13:38.126184 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:13:38.126164 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:14:35.664104 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:14:35.664059 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:16:52.326958 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:16:52.326908 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:16:53.360417 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:16:53.360383 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:18:31.908977 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:18:31.908881 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:18:38.146818 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:18:38.146789 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:18:38.150073 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:18:38.150051 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:19:20.411982 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:19:20.411948 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:23:38.168957 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:23:38.168925 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:23:38.173100 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:23:38.173077 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:24:25.975989 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:25.975957 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:27.785523 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785492 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pgghz/must-gather-nbsws"] Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785794 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="storage-initializer" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785805 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="storage-initializer" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785814 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kube-rbac-proxy" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785820 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kube-rbac-proxy" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785830 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785836 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785844 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kube-rbac-proxy" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785850 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kube-rbac-proxy" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785857 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="storage-initializer" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785861 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="storage-initializer" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785871 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" Jul 16 18:24:27.785883 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785877 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" Jul 16 18:24:27.786261 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785923 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kserve-container" Jul 16 18:24:27.786261 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785930 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kserve-container" Jul 16 18:24:27.786261 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785936 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d10dbe18-5da2-481f-9579-f985ccb3e492" containerName="kube-rbac-proxy" Jul 16 18:24:27.786261 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.785942 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf50dfb-21f7-4f35-8496-a8cfaa013804" containerName="kube-rbac-proxy" Jul 16 18:24:27.788610 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.788595 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:27.790992 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.790969 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pgghz\"/\"kube-root-ca.crt\"" Jul 16 18:24:27.792035 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.792004 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pgghz\"/\"openshift-service-ca.crt\"" Jul 16 18:24:27.792137 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.792050 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pgghz\"/\"default-dockercfg-rc68n\"" Jul 16 18:24:27.797013 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.796993 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pgghz/must-gather-nbsws"] Jul 16 18:24:27.832956 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.832935 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:27.906336 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.906304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww2vd\" (UniqueName: \"kubernetes.io/projected/5038f19a-3824-42b1-ab6f-99f444d84e65-kube-api-access-ww2vd\") pod \"must-gather-nbsws\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:27.906484 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:27.906341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5038f19a-3824-42b1-ab6f-99f444d84e65-must-gather-output\") pod \"must-gather-nbsws\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:28.007636 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.007598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww2vd\" (UniqueName: \"kubernetes.io/projected/5038f19a-3824-42b1-ab6f-99f444d84e65-kube-api-access-ww2vd\") pod \"must-gather-nbsws\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:28.007775 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.007642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5038f19a-3824-42b1-ab6f-99f444d84e65-must-gather-output\") pod \"must-gather-nbsws\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:28.008014 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.007992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5038f19a-3824-42b1-ab6f-99f444d84e65-must-gather-output\") pod \"must-gather-nbsws\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:28.017711 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.017688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww2vd\" (UniqueName: \"kubernetes.io/projected/5038f19a-3824-42b1-ab6f-99f444d84e65-kube-api-access-ww2vd\") pod \"must-gather-nbsws\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:28.098443 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.098365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:24:28.226157 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.226128 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pgghz/must-gather-nbsws"] Jul 16 18:24:28.228636 ip-10-0-141-151 kubenswrapper[2579]: W0716 18:24:28.228607 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5038f19a_3824_42b1_ab6f_99f444d84e65.slice/crio-9affe5c0cbbe8c6f7651695cef2f78acfa52310748fd64a5501f69b52272dc78 WatchSource:0}: Error finding container 9affe5c0cbbe8c6f7651695cef2f78acfa52310748fd64a5501f69b52272dc78: Status 404 returned error can't find the container with id 9affe5c0cbbe8c6f7651695cef2f78acfa52310748fd64a5501f69b52272dc78 Jul 16 18:24:28.230354 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.230335 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 18:24:28.461576 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:28.461542 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pgghz/must-gather-nbsws" event={"ID":"5038f19a-3824-42b1-ab6f-99f444d84e65","Type":"ContainerStarted","Data":"9affe5c0cbbe8c6f7651695cef2f78acfa52310748fd64a5501f69b52272dc78"} Jul 16 18:24:35.496846 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:35.496800 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pgghz/must-gather-nbsws" event={"ID":"5038f19a-3824-42b1-ab6f-99f444d84e65","Type":"ContainerStarted","Data":"c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe"} Jul 16 18:24:35.497331 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:35.496860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pgghz/must-gather-nbsws" event={"ID":"5038f19a-3824-42b1-ab6f-99f444d84e65","Type":"ContainerStarted","Data":"c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12"} Jul 16 18:24:35.517705 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:35.517652 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pgghz/must-gather-nbsws" podStartSLOduration=2.371315691 podStartE2EDuration="8.517637339s" totalImagesPullingTime="6.146321648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 18:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:24:35.516412715 +0000 UTC m=+3058.002705753" watchObservedRunningTime="2026-07-16 18:24:35.517637339 +0000 UTC m=+3058.003930345" Jul 16 18:24:43.553225 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:43.553192 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:43.654310 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:43.654280 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:58.599075 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:58.599010 2579 generic.go:356] "Generic (PLEG): container finished" podID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerID="c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12" exitCode=0 Jul 16 18:24:58.599523 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:58.599088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pgghz/must-gather-nbsws" event={"ID":"5038f19a-3824-42b1-ab6f-99f444d84e65","Type":"ContainerDied","Data":"c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12"} Jul 16 18:24:58.599523 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:58.599395 2579 scope.go:117] "RemoveContainer" containerID="c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12" Jul 16 18:24:58.637669 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:58.637648 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pgghz_must-gather-nbsws_5038f19a-3824-42b1-ab6f-99f444d84e65/gather/0.log" Jul 16 18:24:59.116677 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:59.116649 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:59.259412 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:59.259357 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:59.324713 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:24:59.324686 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:25:01.967627 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:01.967591 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4tq6n_62952831-854a-48e2-886c-3cdf2492c97a/global-pull-secret-syncer/0.log" Jul 16 18:25:02.105457 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:02.105423 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" Jul 16 18:25:02.261342 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:02.261269 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-151.ec2.internal_cd1a81f81de33839e1fd7043ad57b780/haproxy/0.log" Jul 16 18:25:04.082884 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.082855 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-pgghz/must-gather-nbsws"] Jul 16 18:25:04.083314 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.083056 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-pgghz/must-gather-nbsws" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="copy" containerID="cri-o://c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe" gracePeriod=2 Jul 16 18:25:04.087488 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.087462 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-pgghz/must-gather-nbsws"] Jul 16 18:25:04.318582 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.318559 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pgghz_must-gather-nbsws_5038f19a-3824-42b1-ab6f-99f444d84e65/copy/0.log" Jul 16 18:25:04.320314 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.320296 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:25:04.322373 ip-10-0-141-151 kubenswrapper[2579]: E0716 18:25:04.322349 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nbsws\" is forbidden: User \"system:node:ip-10-0-141-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-pgghz\": no relationship found between node 'ip-10-0-141-151.ec2.internal' and this object" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:25:04.329728 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.329711 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww2vd\" (UniqueName: \"kubernetes.io/projected/5038f19a-3824-42b1-ab6f-99f444d84e65-kube-api-access-ww2vd\") pod \"5038f19a-3824-42b1-ab6f-99f444d84e65\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " Jul 16 18:25:04.329802 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.329740 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5038f19a-3824-42b1-ab6f-99f444d84e65-must-gather-output\") pod \"5038f19a-3824-42b1-ab6f-99f444d84e65\" (UID: \"5038f19a-3824-42b1-ab6f-99f444d84e65\") " Jul 16 18:25:04.330965 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.330939 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5038f19a-3824-42b1-ab6f-99f444d84e65-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "5038f19a-3824-42b1-ab6f-99f444d84e65" (UID: "5038f19a-3824-42b1-ab6f-99f444d84e65"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:25:04.331979 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.331956 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5038f19a-3824-42b1-ab6f-99f444d84e65-kube-api-access-ww2vd" (OuterVolumeSpecName: "kube-api-access-ww2vd") pod "5038f19a-3824-42b1-ab6f-99f444d84e65" (UID: "5038f19a-3824-42b1-ab6f-99f444d84e65"). InnerVolumeSpecName "kube-api-access-ww2vd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:25:04.430810 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.430764 2579 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5038f19a-3824-42b1-ab6f-99f444d84e65-must-gather-output\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 18:25:04.430810 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.430804 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww2vd\" (UniqueName: \"kubernetes.io/projected/5038f19a-3824-42b1-ab6f-99f444d84e65-kube-api-access-ww2vd\") on node \"ip-10-0-141-151.ec2.internal\" DevicePath \"\"" Jul 16 18:25:04.622567 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.622498 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-pgghz_must-gather-nbsws_5038f19a-3824-42b1-ab6f-99f444d84e65/copy/0.log" Jul 16 18:25:04.622868 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.622848 2579 generic.go:356] "Generic (PLEG): container finished" podID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerID="c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe" exitCode=143 Jul 16 18:25:04.622943 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.622893 2579 scope.go:117] "RemoveContainer" containerID="c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe" Jul 16 18:25:04.622943 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.622896 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:25:04.625225 ip-10-0-141-151 kubenswrapper[2579]: E0716 18:25:04.625198 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nbsws\" is forbidden: User \"system:node:ip-10-0-141-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-pgghz\": no relationship found between node 'ip-10-0-141-151.ec2.internal' and this object" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:25:04.630786 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.630767 2579 scope.go:117] "RemoveContainer" containerID="c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12" Jul 16 18:25:04.632975 ip-10-0-141-151 kubenswrapper[2579]: E0716 18:25:04.632946 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nbsws\" is forbidden: User \"system:node:ip-10-0-141-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-pgghz\": no relationship found between node 'ip-10-0-141-151.ec2.internal' and this object" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" pod="openshift-must-gather-pgghz/must-gather-nbsws" Jul 16 18:25:04.641967 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.641943 2579 scope.go:117] "RemoveContainer" containerID="c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe" Jul 16 18:25:04.642218 ip-10-0-141-151 kubenswrapper[2579]: E0716 18:25:04.642201 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe\": container with ID starting with c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe not found: ID does not exist" containerID="c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe" Jul 16 18:25:04.642268 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.642226 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe"} err="failed to get container status \"c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe\": rpc error: code = NotFound desc = could not find container \"c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe\": container with ID starting with c7d2c1b7180c52a4a68fd3d74625a93d909ef59a5eed0654cef7b2847725a4fe not found: ID does not exist" Jul 16 18:25:04.642268 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.642242 2579 scope.go:117] "RemoveContainer" containerID="c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12" Jul 16 18:25:04.642487 ip-10-0-141-151 kubenswrapper[2579]: E0716 18:25:04.642469 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12\": container with ID starting with c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12 not found: ID does not exist" containerID="c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12" Jul 16 18:25:04.642523 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:04.642496 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12"} err="failed to get container status \"c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12\": rpc error: code = NotFound desc = could not find container \"c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12\": container with ID starting with c20ec808f6d9ab9d30ebf26ab8bba75f3589130ae25a0831e5cf85e88cfe4e12 not found: ID does not exist" Jul 16 18:25:05.119231 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:05.119196 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:25:05.219402 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:05.219369 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2mbhd_f2dd25ad-b133-41cd-af46-3c7b865ea400/konnectivity-agent/0.log" limit=1024 Jul 16 18:25:06.063598 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.063570 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" path="/var/lib/kubelet/pods/5038f19a-3824-42b1-ab6f-99f444d84e65/volumes" Jul 16 18:25:06.535485 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.535459 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-826js_1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533/kube-state-metrics/0.log" Jul 16 18:25:06.559150 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.559122 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-826js_1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533/kube-rbac-proxy-main/0.log" Jul 16 18:25:06.585797 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.585774 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-826js_1c45d6a7-bcc0-4f59-bb4a-3cc3629cb533/kube-rbac-proxy-self/0.log" Jul 16 18:25:06.650254 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.650227 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-gbxbq_d9e40ed8-6afe-473d-b3c8-00c98930c2a6/monitoring-plugin/0.log" Jul 16 18:25:06.682721 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.682698 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b82tg_485e738c-fcf1-4b9a-a462-d59721e1bcba/node-exporter/0.log" Jul 16 18:25:06.707812 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.707788 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b82tg_485e738c-fcf1-4b9a-a462-d59721e1bcba/kube-rbac-proxy/0.log" Jul 16 18:25:06.732206 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.732187 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b82tg_485e738c-fcf1-4b9a-a462-d59721e1bcba/init-textfile/0.log" Jul 16 18:25:06.957990 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.957967 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-rt4zp_a25478d6-c0b0-45eb-8bf4-0aa93160747c/kube-rbac-proxy-main/0.log" Jul 16 18:25:06.986599 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:06.986574 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-rt4zp_a25478d6-c0b0-45eb-8bf4-0aa93160747c/kube-rbac-proxy-self/0.log" Jul 16 18:25:07.011903 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.011879 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-rt4zp_a25478d6-c0b0-45eb-8bf4-0aa93160747c/openshift-state-metrics/0.log" Jul 16 18:25:07.060983 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.060958 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/prometheus/0.log" Jul 16 18:25:07.084608 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.084579 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/config-reloader/0.log" Jul 16 18:25:07.116098 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.116077 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/thanos-sidecar/0.log" Jul 16 18:25:07.140788 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.140772 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/kube-rbac-proxy-web/0.log" Jul 16 18:25:07.168311 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.168249 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/kube-rbac-proxy/0.log" Jul 16 18:25:07.196693 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.196672 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/kube-rbac-proxy-thanos/0.log" Jul 16 18:25:07.229275 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:07.229256 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_89711504-4ff1-49bd-984b-c3ad92830d00/init-config-reloader/0.log" Jul 16 18:25:09.668977 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.668934 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs"] Jul 16 18:25:09.669345 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.669279 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="copy" Jul 16 18:25:09.669345 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.669296 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="copy" Jul 16 18:25:09.669428 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.669343 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="gather" Jul 16 18:25:09.669428 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.669354 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="gather" Jul 16 18:25:09.669428 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.669426 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="copy" Jul 16 18:25:09.669571 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.669435 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5038f19a-3824-42b1-ab6f-99f444d84e65" containerName="gather" Jul 16 18:25:09.673571 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.673550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.676238 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.676216 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hpk2v\"/\"kube-root-ca.crt\"" Jul 16 18:25:09.677244 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.677225 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hpk2v\"/\"default-dockercfg-g2x2f\"" Jul 16 18:25:09.677319 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.677275 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hpk2v\"/\"openshift-service-ca.crt\"" Jul 16 18:25:09.682732 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.682712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs"] Jul 16 18:25:09.773079 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.773048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-podres\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.773239 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.773104 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-proc\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.773239 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.773130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-lib-modules\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.773239 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.773197 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-sys\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.773239 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.773226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kdnl\" (UniqueName: \"kubernetes.io/projected/a8805d57-1f1b-43ff-9064-16da45ffa06c-kube-api-access-5kdnl\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873576 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873543 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-sys\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873703 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kdnl\" (UniqueName: \"kubernetes.io/projected/a8805d57-1f1b-43ff-9064-16da45ffa06c-kube-api-access-5kdnl\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873703 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873663 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-sys\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873813 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873712 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-podres\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873813 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-proc\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873813 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-lib-modules\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873944 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873858 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-podres\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.873944 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873890 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-proc\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.874064 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.873975 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8805d57-1f1b-43ff-9064-16da45ffa06c-lib-modules\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.882069 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.882048 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kdnl\" (UniqueName: \"kubernetes.io/projected/a8805d57-1f1b-43ff-9064-16da45ffa06c-kube-api-access-5kdnl\") pod \"perf-node-gather-daemonset-7rwhs\" (UID: \"a8805d57-1f1b-43ff-9064-16da45ffa06c\") " pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:09.984071 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:09.984047 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:10.115272 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:10.115241 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs"] Jul 16 18:25:10.118179 ip-10-0-141-151 kubenswrapper[2579]: W0716 18:25:10.118155 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda8805d57_1f1b_43ff_9064_16da45ffa06c.slice/crio-45dd909a54b7eb2e1544a09af481f287fb4613961ed8a7fc8e577e5e779bea00 WatchSource:0}: Error finding container 45dd909a54b7eb2e1544a09af481f287fb4613961ed8a7fc8e577e5e779bea00: Status 404 returned error can't find the container with id 45dd909a54b7eb2e1544a09af481f287fb4613961ed8a7fc8e577e5e779bea00 Jul 16 18:25:10.647037 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:10.646994 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" event={"ID":"a8805d57-1f1b-43ff-9064-16da45ffa06c","Type":"ContainerStarted","Data":"030161123f2686defefa8af8db8b9b9cf3027fc7e4652be4bcf0c8eba4576815"} Jul 16 18:25:10.647225 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:10.647079 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" event={"ID":"a8805d57-1f1b-43ff-9064-16da45ffa06c","Type":"ContainerStarted","Data":"45dd909a54b7eb2e1544a09af481f287fb4613961ed8a7fc8e577e5e779bea00"} Jul 16 18:25:10.647225 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:10.647130 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:10.664320 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:10.664276 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" podStartSLOduration=1.6642616860000001 podStartE2EDuration="1.664261686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 18:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:25:10.66304773 +0000 UTC m=+3093.149340728" watchObservedRunningTime="2026-07-16 18:25:10.664261686 +0000 UTC m=+3093.150554692" Jul 16 18:25:11.260960 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:11.260919 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m24mr_e4221d61-f6c0-43bd-b6ac-38739ab0097e/dns/0.log" Jul 16 18:25:11.289144 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:11.289119 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m24mr_e4221d61-f6c0-43bd-b6ac-38739ab0097e/kube-rbac-proxy/0.log" Jul 16 18:25:11.405114 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:11.405074 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-72cjz_ddb47c98-02dd-4f7d-8830-d9f56f73f9e3/dns-node-resolver/0.log" Jul 16 18:25:12.006168 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:12.006137 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rsl6_1a335436-5134-4780-95ff-b3d182fb2969/node-ca/0.log" Jul 16 18:25:13.164132 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:13.164103 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dj4t_3615bce5-3d4d-41a5-b5dc-4e460c50ba80/serve-healthcheck-canary/0.log" Jul 16 18:25:13.711615 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:13.711587 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tcfbb_df77e394-be0a-46a6-b2cc-90ff91090c9d/kube-rbac-proxy/0.log" Jul 16 18:25:13.734227 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:13.734205 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tcfbb_df77e394-be0a-46a6-b2cc-90ff91090c9d/exporter/0.log" Jul 16 18:25:13.757479 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:13.757457 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tcfbb_df77e394-be0a-46a6-b2cc-90ff91090c9d/extractor/0.log" Jul 16 18:25:16.013201 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:16.013170 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tglc2_2df27344-a459-4c33-89ce-82626795f04d/s3-init/0.log" Jul 16 18:25:16.029059 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:16.029020 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tglc2_2df27344-a459-4c33-89ce-82626795f04d/download-hf-model/0.log" Jul 16 18:25:16.660197 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:16.660169 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hpk2v/perf-node-gather-daemonset-7rwhs" Jul 16 18:25:20.661184 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:20.661144 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-stvkn_6c7131e0-7ce7-42a3-8707-b8af0419d0f7/kube-storage-version-migrator-operator/1.log" Jul 16 18:25:20.662242 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:20.662221 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-stvkn_6c7131e0-7ce7-42a3-8707-b8af0419d0f7/kube-storage-version-migrator-operator/0.log" Jul 16 18:25:21.902335 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:21.902304 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/kube-multus-additional-cni-plugins/0.log" Jul 16 18:25:21.924376 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:21.924351 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/egress-router-binary-copy/0.log" Jul 16 18:25:21.951303 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:21.951279 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/cni-plugins/0.log" Jul 16 18:25:21.975301 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:21.975281 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/bond-cni-plugin/0.log" Jul 16 18:25:21.999835 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:21.999818 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/routeoverride-cni/0.log" Jul 16 18:25:22.026189 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:22.026166 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/whereabouts-cni-bincopy/0.log" Jul 16 18:25:22.051957 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:22.051931 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ncx2q_04283751-84b7-4886-8a2e-e7e3085d2377/whereabouts-cni/0.log" Jul 16 18:25:22.081813 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:22.081773 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h8kcl_16e7f2ae-f5aa-4116-a508-eaf0787a813a/kube-multus/0.log" Jul 16 18:25:22.291338 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:22.291309 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nm4p2_84b3a525-e62d-44eb-b93f-2c8d1873420c/network-metrics-daemon/0.log" Jul 16 18:25:22.312606 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:22.312583 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nm4p2_84b3a525-e62d-44eb-b93f-2c8d1873420c/kube-rbac-proxy/0.log" Jul 16 18:25:23.335497 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.335465 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-controller/0.log" Jul 16 18:25:23.355000 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.354973 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/0.log" Jul 16 18:25:23.355000 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.355002 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovn-acl-logging/1.log" Jul 16 18:25:23.372725 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.372707 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/kube-rbac-proxy-node/0.log" Jul 16 18:25:23.396725 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.396700 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/kube-rbac-proxy-ovn-metrics/0.log" Jul 16 18:25:23.417468 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.417450 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/northd/0.log" Jul 16 18:25:23.439395 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.439374 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/nbdb/0.log" Jul 16 18:25:23.461261 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.461241 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/sbdb/0.log" Jul 16 18:25:23.504651 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:23.504627 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jcjq6_8e74d041-c34c-4f23-8952-a747e8c312a2/ovnkube-controller/0.log" Jul 16 18:25:24.706228 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:24.706201 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-b87sw_d3fa8548-aca3-4884-9081-b61951f36844/network-check-target-container/0.log" Jul 16 18:25:25.732922 ip-10-0-141-151 kubenswrapper[2579]: I0716 18:25:25.732899 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-tcbhq_380ce908-8115-4e31-9216-4ffded74c95e/iptables-alerter/0.log"