Jul 08 16:56:28.121465 ip-10-0-143-156 systemd[1]: Starting Kubernetes Kubelet... Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.509325 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512955 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512968 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512973 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512976 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512980 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512984 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:56:28.644673 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512992 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:56:28.586000 ip-10-0-143-156 systemd[1]: Started Kubernetes Kubelet. Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.512995 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513007 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513010 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513013 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513016 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513019 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513022 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513024 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513027 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513030 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513032 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513035 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513038 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513041 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513044 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513046 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513049 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513051 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513054 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:56:28.645627 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513058 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513061 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513064 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513067 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513069 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513071 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513074 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513076 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513079 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513082 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513084 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513087 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513091 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513094 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513097 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513099 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513102 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513105 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513108 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:56:28.646480 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513110 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513113 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513115 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513118 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513121 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513123 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513126 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513129 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513131 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513134 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513137 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513139 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513155 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513159 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513162 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513164 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513167 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513169 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513172 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513175 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:56:28.647185 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513178 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513180 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513183 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513186 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513188 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513191 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513195 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513199 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513202 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513205 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513208 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513210 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513213 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513216 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513218 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513221 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513224 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513227 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513230 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513232 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:56:28.647943 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513236 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513238 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513241 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513243 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513246 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513248 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513251 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513253 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513256 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513258 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513261 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513264 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513266 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.513268 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514017 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514023 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514027 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514029 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514032 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514035 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:56:28.648633 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514038 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514041 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514043 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514047 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514050 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514052 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514055 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514058 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514061 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514064 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514066 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514069 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514072 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514074 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514077 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514080 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514083 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514086 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514088 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514091 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:56:28.649406 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514093 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514096 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514098 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514101 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514103 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514106 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514108 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514111 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514114 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514116 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514119 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514122 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514125 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514128 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514130 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514133 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514136 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514139 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514164 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514169 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:56:28.650210 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514173 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514177 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514181 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514184 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514187 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514190 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514192 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514195 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514197 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514202 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514206 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514209 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514212 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514215 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514218 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514221 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514224 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514226 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514228 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:56:28.650949 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514232 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514234 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514237 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514239 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514242 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514246 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514249 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514252 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514254 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514256 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514259 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514262 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514264 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514267 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514270 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514273 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514276 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514279 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514281 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:56:28.651719 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514284 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514287 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514290 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514296 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514298 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514301 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514303 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514306 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514309 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514311 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514313 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514317 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514320 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514323 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514326 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514712 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514717 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514720 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514723 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:56:28.652490 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514728 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514731 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514734 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514736 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514739 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514741 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514744 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514747 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514749 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514753 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514756 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514759 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514762 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514765 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514768 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514770 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514774 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514777 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514780 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:56:28.653306 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514783 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514785 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514788 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514790 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514793 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514796 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514798 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514801 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514803 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514805 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514808 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514810 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514813 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514817 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514819 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514822 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514825 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514827 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514831 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514833 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:56:28.654050 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514836 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514839 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514841 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514844 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514846 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514849 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514851 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514854 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514856 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514860 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514863 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514865 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514868 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514871 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514873 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514877 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514879 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514882 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514884 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514887 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:56:28.654725 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514889 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514892 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514894 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514897 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514899 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514904 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514907 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514910 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514912 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514915 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514917 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514920 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514922 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514925 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514927 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514929 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514932 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514934 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514938 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514941 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:56:28.655348 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514944 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514948 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514950 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514953 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514957 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514960 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514963 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514966 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514969 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514972 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514975 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514977 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514979 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514982 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514984 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.514987 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515060 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515070 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515076 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515082 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515086 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 16:56:28.655963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515089 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515093 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515098 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515101 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515104 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515108 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515111 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515114 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515117 2563 flags.go:64] FLAG: --cgroup-root="" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515120 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515123 2563 flags.go:64] FLAG: --client-ca-file="" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515125 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515130 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515134 2563 flags.go:64] FLAG: --cluster-domain="" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515137 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515140 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515157 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515161 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515165 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515168 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515171 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515174 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515177 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515180 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 16:56:28.656624 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515183 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515186 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515189 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515193 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515197 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515200 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515203 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515207 2563 flags.go:64] FLAG: --enable-server="true" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515210 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515215 2563 flags.go:64] FLAG: --event-burst="100" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515218 2563 flags.go:64] FLAG: --event-qps="50" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515221 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515224 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515227 2563 flags.go:64] FLAG: --eviction-hard="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515230 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515233 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515236 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515239 2563 flags.go:64] FLAG: --eviction-soft="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515242 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515245 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515249 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515252 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515255 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515258 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515261 2563 flags.go:64] FLAG: --feature-gates="" Jul 08 16:56:28.657557 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515265 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515269 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515272 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515275 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515278 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515281 2563 flags.go:64] FLAG: --help="false" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515284 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-143-156.ec2.internal" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515287 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515289 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515292 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515295 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515301 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515304 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515307 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515310 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515313 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515316 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515320 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515323 2563 flags.go:64] FLAG: --kube-reserved="" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515326 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515328 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515331 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515334 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515337 2563 flags.go:64] FLAG: --lock-file="" Jul 08 16:56:28.658465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515339 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515342 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515345 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515352 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515355 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515358 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515360 2563 flags.go:64] FLAG: --logging-format="text" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515363 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515367 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515370 2563 flags.go:64] FLAG: --manifest-url="" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515373 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515377 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515380 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515384 2563 flags.go:64] FLAG: --max-pods="110" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515387 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515390 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515393 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515396 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515399 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515403 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515406 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515413 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515417 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515419 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 16:56:28.660735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515423 2563 flags.go:64] FLAG: --pod-cidr="" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515426 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515432 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515435 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515438 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515441 2563 flags.go:64] FLAG: --port="10250" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515444 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515447 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-081baa382d5899e95" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515450 2563 flags.go:64] FLAG: --qos-reserved="" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515453 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515456 2563 flags.go:64] FLAG: --register-node="true" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515460 2563 flags.go:64] FLAG: --register-with-taints="" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515468 2563 flags.go:64] FLAG: --registry-burst="10" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515471 2563 flags.go:64] FLAG: --registry-qps="5" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515474 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515477 2563 flags.go:64] FLAG: --reserved-memory="" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515481 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515484 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515487 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515490 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515493 2563 flags.go:64] FLAG: --runonce="false" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515496 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515499 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515502 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515505 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 16:56:28.661511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515507 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515511 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515514 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515518 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515521 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515524 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515527 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515530 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515534 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515536 2563 flags.go:64] FLAG: --system-cgroups="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515539 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515544 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515547 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515550 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515554 2563 flags.go:64] FLAG: --tls-min-version="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515557 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515560 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515562 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515566 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515569 2563 flags.go:64] FLAG: --v="2" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515573 2563 flags.go:64] FLAG: --version="false" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515577 2563 flags.go:64] FLAG: --vmodule="" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515580 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.515584 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515699 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:56:28.903972 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515702 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515706 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515709 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515711 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515714 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515717 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515720 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515723 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515726 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515729 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515733 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515736 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515738 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515741 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515744 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515746 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515749 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515752 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515756 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:56:28.921194 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515759 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515762 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515765 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515767 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515770 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515773 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515776 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515779 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515781 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515784 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515786 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515789 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515792 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515794 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515797 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515800 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515802 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515804 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515807 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515810 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:56:28.921835 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515812 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515815 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515817 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515821 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515823 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515826 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515830 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515833 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515836 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515839 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515842 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515844 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515847 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515849 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515852 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515855 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515857 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515859 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515863 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515866 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:56:28.922525 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515868 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515871 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515873 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515876 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515878 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515881 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515883 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515886 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515888 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515891 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515893 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515896 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515898 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515901 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515903 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515908 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515910 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515914 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515917 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515920 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:56:28.923115 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515923 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515926 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515929 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515932 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515935 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515937 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515940 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515942 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515945 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515947 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515951 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515954 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515956 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515959 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515962 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515964 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515967 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515969 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:56:28.923785 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.515972 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.516820 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.525134 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.525168 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.526779 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.526788 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526849 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526855 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526859 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526862 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526866 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526869 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526872 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526895 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526899 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:56:28.924347 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526902 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526906 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526909 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526912 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526915 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526918 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526921 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526924 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526927 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526929 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526932 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526934 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526937 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526941 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526945 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526948 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526951 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526953 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526956 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:56:28.924886 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526959 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526962 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526965 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526968 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526971 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526973 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526976 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526978 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526981 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526986 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526989 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526993 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526995 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.526998 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527001 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527004 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527007 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527009 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527012 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527015 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:56:28.925486 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527018 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527020 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527023 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527025 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527028 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527031 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527033 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527036 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527038 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527041 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527044 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527046 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527049 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527052 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527054 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527057 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527060 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527062 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527065 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527067 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:56:28.926190 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527070 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527074 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527076 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527081 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527083 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527086 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527088 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527091 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527094 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527096 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527099 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527103 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527106 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527108 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527111 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527114 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527116 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527119 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527121 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:56:28.926845 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527124 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527128 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527132 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527134 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527138 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527157 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527161 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527164 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527167 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527170 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527173 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527176 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.527181 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527302 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 16:56:28.927473 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527307 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527312 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527316 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527319 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527321 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527324 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527327 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527329 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527332 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527334 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527337 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527345 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527348 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527350 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527353 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527355 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527359 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527364 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527367 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 16:56:28.927907 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527371 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527373 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527376 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527379 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527382 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527384 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527387 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527389 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527392 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527394 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527397 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527400 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527402 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527405 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527408 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527413 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527415 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527418 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527420 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527423 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 16:56:28.928566 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527426 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527428 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527431 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527433 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527436 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527439 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527442 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527444 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527447 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527449 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527452 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527455 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527457 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527460 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527462 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527465 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527467 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527470 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527474 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527477 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 16:56:28.929163 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527480 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527483 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527486 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527489 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527492 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527494 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527498 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527502 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527505 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527508 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527510 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527513 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527516 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527518 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527521 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527523 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527526 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527528 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527531 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527534 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 16:56:28.929808 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527536 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527539 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527541 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527544 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527547 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527549 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527552 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527554 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527557 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527559 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527562 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527564 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527567 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527569 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527572 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527574 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527577 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527579 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 16:56:28.930401 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:28.527584 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.527590 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.527713 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.530883 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.532206 2563 server.go:1019] "Starting client certificate rotation" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.532313 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.532349 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.534615 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.539117 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.549584 2563 log.go:25] "Validated CRI v1 runtime API" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.552305 2563 log.go:25] "Validated CRI v1 image API" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.553060 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 16:56:28.930982 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.555346 2563 fs.go:135] Filesystem UUIDs: map[1440b871-02c3-4fa1-9cbc-d31a5673e100:/dev/nvme0n1p4 40d5d377-a0ac-4631-97bb-97a3cb0e0a12:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 08 16:56:28.931383 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.555361 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.560688 2563 manager.go:216] Machine: {Timestamp:2026-07-08 16:56:28.559947077 +0000 UTC m=+0.322938929 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100143 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21c3a45b65ed777f31d1f8e8141059 SystemUUID:ec21c3a4-5b65-ed77-7f31-d1f8e8141059 BootID:cd507762-2e04-4246-9814-c68c74b36e54 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:35:f2:9e:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:35:f2:9e:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:dd:92:8c:91:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.560821 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.560917 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.563256 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.563369 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.563399 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-156.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 16:56:29.131639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.564716 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.564730 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.564738 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.565418 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.566749 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569646 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569773 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569808 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569825 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569840 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569849 2563 kubelet.go:398] "Adding apiserver pod source" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.569857 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.570333 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.570349 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.572934 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.578785 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.578830 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580163 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580178 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580183 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580189 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580197 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580203 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580211 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580216 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580223 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580229 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580237 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.580243 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.581625 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.581661 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.581763 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.581773 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.584087 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-156.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.585202 2563 server.go:1268] "Started kubelet" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.585284 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.585364 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.585919 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.587288 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-snsnn" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.587975 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 08 16:56:29.133620 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.589790 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.592463 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-156.ec2.internal.18c05ee4a6f26c86 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-156.ec2.internal,UID:ip-10-0-143-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-156.ec2.internal,},FirstTimestamp:2026-07-08 16:56:28.585176198 +0000 UTC m=+0.348168050,LastTimestamp:2026-07-08 16:56:28.585176198 +0000 UTC m=+0.348168050,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-156.ec2.internal,}" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.595242 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.595247 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.595362 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.595522 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-snsnn" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.596079 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.596080 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.596099 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.596230 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.596241 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.597258 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.597285 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.599125 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599297 2563 factory.go:55] Registering systemd factory Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599315 2563 factory.go:223] Registration of the systemd container factory successfully Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599509 2563 factory.go:153] Registering CRI-O factory Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599522 2563 factory.go:223] Registration of the crio container factory successfully Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599566 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599594 2563 factory.go:103] Registering Raw factory Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599605 2563 manager.go:1196] Started watching for new ooms in manager Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.599952 2563 manager.go:349] Starting recovery of all containers Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.615757 2563 manager.go:354] Recovery completed Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.617558 2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.621730 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.626721 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.626751 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.626767 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.627325 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.627334 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.627353 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.627389 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.134865 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.628045 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.628068 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.628080 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.628648 2563 policy_none.go:49] "None policy: Start" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.628665 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.628676 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.629707 2563 policy_none.go:47] "Start" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.669970 2563 manager.go:337] "Starting Device Plugin manager" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.670144 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.670155 2563 server.go:87] "Starting device plugin registration server" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.670511 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.670525 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.670630 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.670702 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.670709 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.671300 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.671343 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.776250 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.777412 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.777446 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.777461 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.777498 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.786410 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.786435 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-156.ec2.internal\": node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.788662 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.789948 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.789984 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.790012 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.790049 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.792568 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.835947 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.894475 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal"] Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.894590 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.896238 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.896295 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.896306 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.897608 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.897721 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.136665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.897754 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.898369 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.898376 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.898395 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.898398 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.898441 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.898409 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.899448 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0052804689573a175003f01e231a790e-config\") pod \"kube-apiserver-proxy-ip-10-0-143-156.ec2.internal\" (UID: \"0052804689573a175003f01e231a790e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.899778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.899801 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.903612 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientMemory" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.903641 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.903654 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeHasSufficientPID" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.936006 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.936691 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-156.ec2.internal\" not found" node="ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:28.941965 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-156.ec2.internal\" not found" node="ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.999616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec3d18bceb905803cc4190ce836dffb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal\" (UID: \"ec3d18bceb905803cc4190ce836dffb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.999664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0052804689573a175003f01e231a790e-config\") pod \"kube-apiserver-proxy-ip-10-0-143-156.ec2.internal\" (UID: \"0052804689573a175003f01e231a790e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.999684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec3d18bceb905803cc4190ce836dffb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal\" (UID: \"ec3d18bceb905803cc4190ce836dffb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:28.999751 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0052804689573a175003f01e231a790e-config\") pod \"kube-apiserver-proxy-ip-10-0-143-156.ec2.internal\" (UID: \"0052804689573a175003f01e231a790e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.037139 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.357995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.100094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec3d18bceb905803cc4190ce836dffb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal\" (UID: \"ec3d18bceb905803cc4190ce836dffb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.100125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec3d18bceb905803cc4190ce836dffb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal\" (UID: \"ec3d18bceb905803cc4190ce836dffb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.100194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec3d18bceb905803cc4190ce836dffb9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal\" (UID: \"ec3d18bceb905803cc4190ce836dffb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.100213 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec3d18bceb905803cc4190ce836dffb9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal\" (UID: \"ec3d18bceb905803cc4190ce836dffb9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.138011 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.240424 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.240446 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.245985 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.359445 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.343537 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.583952 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.444060 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.583952 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.531500 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 16:56:29.583952 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.531704 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 16:56:29.583952 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.544486 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.788527 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.597243 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 16:51:28 +0000 UTC" deadline="2027-12-04 07:20:19.71967517 +0000 UTC" Jul 08 16:56:29.788527 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.597274 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12326h23m50.122404989s" Jul 08 16:56:29.788527 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:29.646929 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-156.ec2.internal\" not found" Jul 08 16:56:29.788527 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.704464 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:56:29.788527 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.724733 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.796480 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.809724 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.810792 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.824000 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:29.893822 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3d18bceb905803cc4190ce836dffb9.slice/crio-84f7ea1b433ff3dd54a00836585d679f1246ede18a765116e3a4b2ac0aca4e50 WatchSource:0}: Error finding container 84f7ea1b433ff3dd54a00836585d679f1246ede18a765116e3a4b2ac0aca4e50: Status 404 returned error can't find the container with id 84f7ea1b433ff3dd54a00836585d679f1246ede18a765116e3a4b2ac0aca4e50 Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:29.894174 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0052804689573a175003f01e231a790e.slice/crio-4e35b9ec5ab4f40ee480968dbb98d5e40f9703b84ee40270eae4a72af92b745e WatchSource:0}: Error finding container 4e35b9ec5ab4f40ee480968dbb98d5e40f9703b84ee40270eae4a72af92b745e: Status 404 returned error can't find the container with id 4e35b9ec5ab4f40ee480968dbb98d5e40f9703b84ee40270eae4a72af92b745e Jul 08 16:56:29.995298 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:29.900187 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 16:56:30.203822 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.167501 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:56:30.628823 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.620627 2563 apiserver.go:52] "Watching apiserver" Jul 08 16:56:30.628823 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.620658 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 16:56:30.628823 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.623215 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 16:56:30.628823 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.627671 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 16:56:30.628823 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.628147 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-4dtj9","openshift-multus/multus-5bxrg","openshift-multus/multus-additional-cni-plugins-qkvzz","openshift-multus/network-metrics-daemon-nkwpj","openshift-network-operator/iptables-alerter-f78g2","kube-system/konnectivity-agent-bb8gf","openshift-dns/node-resolver-bc62n","openshift-image-registry/node-ca-7tb8p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal","openshift-network-diagnostics/network-check-target-pb29q","openshift-ovn-kubernetes/ovnkube-node-px95b","kube-system/global-pull-secret-syncer-lgbxp","kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf"] Jul 08 16:56:30.631663 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.630003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.631663 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.631377 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 16:56:30.634502 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.631885 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.634502 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.632571 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.634502 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.632641 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jdbf6\"" Jul 08 16:56:30.634502 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.632571 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.634502 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.633542 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:30.634502 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.633600 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.634605 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.634930 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.635165 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.635370 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.635551 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4z5fx\"" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.636060 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.636221 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.637988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.636696 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.638708 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.638219 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.638897 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.638898 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.639592 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.640114 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.640217 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d7v5j\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.640358 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkp6t\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.640558 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.641334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.641827 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.642396 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.643077 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.643508 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cbht6\"" Jul 08 16:56:30.643639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.643612 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.644429 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.643818 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.644429 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.644152 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4fb7n\"" Jul 08 16:56:30.647955 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.644896 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.647955 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.647117 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 16:56:30.647955 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.647237 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4vj5k\"" Jul 08 16:56:30.647955 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.647418 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:30.647955 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.647547 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:30.650124 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.649716 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.652310 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d6vv9\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.652400 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.652647 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.652703 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.652712 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.652935 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.652939 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.653047 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.653659 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.653808 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.656789 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.656892 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.656935 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rtxdk\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.657337 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.673751 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xn2w" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.682690 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xn2w" Jul 08 16:56:30.721859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.697148 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.721960 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysconfig\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.721998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-systemd\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722025 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f5r8\" (UniqueName: \"kubernetes.io/projected/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-kube-api-access-5f5r8\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722071 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxfmj\" (UniqueName: \"kubernetes.io/projected/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-kube-api-access-mxfmj\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722095 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-socket-dir-parent\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-etc-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-hosts-file\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722179 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-ovnkube-config\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-device-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722235 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-sys-fs\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.724854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r96j4\" (UniqueName: \"kubernetes.io/projected/785527e0-6f25-4c91-88ba-a47aa06d9e94-kube-api-access-r96j4\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722309 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-host\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-cni-multus\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-hostroot\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722452 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-log-socket\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722476 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-cni-netd\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722510 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tffzk\" (UniqueName: \"kubernetes.io/projected/c752ba3a-30e7-4d5a-85f8-eea123b359aa-kube-api-access-tffzk\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-tmp-dir\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722578 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-iptables-alerter-script\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec13312c-4802-428d-bb87-0a70978eb388-dbus\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-cni-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-cni-binary-copy\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-var-lib-kubelet\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.725481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722762 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c562f70-0d3f-4bb9-8092-56fcdefbdb3c-konnectivity-ca\") pod \"konnectivity-agent-bb8gf\" (UID: \"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c\") " pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-cnibin\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722813 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-env-overrides\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722853 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-host-slash\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-kubernetes\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722935 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-run\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0922e014-26df-43c7-93b0-dc3267e0c109-tmp\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.722994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-cni-binary-copy\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723018 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-conf-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723046 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-kubelet\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-var-lib-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-node-log\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-ovnkube-script-lib\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0922e014-26df-43c7-93b0-dc3267e0c109-etc-tuned\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.798617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723250 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-system-cni-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-netns\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723293 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-run-ovn-kubernetes\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87w5l\" (UniqueName: \"kubernetes.io/projected/f6504eba-a3f1-416f-b1df-56308e89d459-kube-api-access-87w5l\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysctl-d\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-sys\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723400 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-daemon-config\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c24715a0-4a60-4813-b316-57aaa69e6247-ovn-node-metrics-cert\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723471 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723496 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-socket-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-registration-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-cni-bin\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723581 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-kubelet\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.799957 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vxnw\" (UniqueName: \"kubernetes.io/projected/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-kube-api-access-4vxnw\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-ovn\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-cni-bin\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-system-cni-dir\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723732 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-etc-selinux\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-k8s-cni-cncf-io\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-etc-kubernetes\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723842 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-run-netns\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723902 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-lib-modules\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhfxf\" (UniqueName: \"kubernetes.io/projected/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-kube-api-access-vhfxf\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-systemd-units\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.723978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-slash\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724007 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.800798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724042 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5rn7\" (UniqueName: \"kubernetes.io/projected/c24715a0-4a60-4813-b316-57aaa69e6247-kube-api-access-k5rn7\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-os-release\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724091 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-modprobe-d\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysctl-conf\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724217 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724257 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-cnibin\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkh7x\" (UniqueName: \"kubernetes.io/projected/0922e014-26df-43c7-93b0-dc3267e0c109-kube-api-access-fkh7x\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-os-release\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c752ba3a-30e7-4d5a-85f8-eea123b359aa-serviceca\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-multus-certs\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-systemd\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724538 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c752ba3a-30e7-4d5a-85f8-eea123b359aa-host\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.802529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724580 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec13312c-4802-428d-bb87-0a70978eb388-kubelet-config\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.803357 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.724624 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c562f70-0d3f-4bb9-8092-56fcdefbdb3c-agent-certs\") pod \"konnectivity-agent-bb8gf\" (UID: \"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c\") " pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.803813 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.803628 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" event={"ID":"0052804689573a175003f01e231a790e","Type":"ContainerStarted","Data":"4e35b9ec5ab4f40ee480968dbb98d5e40f9703b84ee40270eae4a72af92b745e"} Jul 08 16:56:30.809244 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.809008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" event={"ID":"ec3d18bceb905803cc4190ce836dffb9","Type":"ContainerStarted","Data":"84f7ea1b433ff3dd54a00836585d679f1246ede18a765116e3a4b2ac0aca4e50"} Jul 08 16:56:30.825497 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825458 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-k8s-cni-cncf-io\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-etc-kubernetes\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-run-netns\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825560 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825579 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-k8s-cni-cncf-io\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825579 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-etc-kubernetes\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825621 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-run-netns\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825586 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-lib-modules\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.825681 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825669 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhfxf\" (UniqueName: \"kubernetes.io/projected/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-kube-api-access-vhfxf\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825695 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-systemd-units\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-slash\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825757 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-lib-modules\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5rn7\" (UniqueName: \"kubernetes.io/projected/c24715a0-4a60-4813-b316-57aaa69e6247-kube-api-access-k5rn7\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-os-release\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825809 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-slash\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-modprobe-d\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysctl-conf\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825861 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-cnibin\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825920 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-os-release\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.826040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825800 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-systemd-units\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkh7x\" (UniqueName: \"kubernetes.io/projected/0922e014-26df-43c7-93b0-dc3267e0c109-kube-api-access-fkh7x\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-os-release\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826021 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c752ba3a-30e7-4d5a-85f8-eea123b359aa-serviceca\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826046 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826055 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysctl-conf\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.825770 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-multus-certs\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-systemd\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826112 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-multus-certs\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826136 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c752ba3a-30e7-4d5a-85f8-eea123b359aa-host\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826151 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-os-release\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826163 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec13312c-4802-428d-bb87-0a70978eb388-kubelet-config\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.826661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826188 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c562f70-0d3f-4bb9-8092-56fcdefbdb3c-agent-certs\") pod \"konnectivity-agent-bb8gf\" (UID: \"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c\") " pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826237 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826254 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysconfig\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.826307 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826341 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-systemd\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.826373 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:31.326355367 +0000 UTC m=+3.089347242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826386 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c752ba3a-30e7-4d5a-85f8-eea123b359aa-host\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826402 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-cnibin\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826437 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec13312c-4802-428d-bb87-0a70978eb388-kubelet-config\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826069 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-modprobe-d\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-systemd\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f5r8\" (UniqueName: \"kubernetes.io/projected/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-kube-api-access-5f5r8\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxfmj\" (UniqueName: \"kubernetes.io/projected/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-kube-api-access-mxfmj\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.827292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-socket-dir-parent\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826634 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c752ba3a-30e7-4d5a-85f8-eea123b359aa-serviceca\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-etc-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-hosts-file\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-ovnkube-config\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-device-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826734 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-sys-fs\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826752 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r96j4\" (UniqueName: \"kubernetes.io/projected/785527e0-6f25-4c91-88ba-a47aa06d9e94-kube-api-access-r96j4\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-host\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-cni-multus\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-hostroot\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826308 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysconfig\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-log-socket\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-cni-netd\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.827903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826861 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tffzk\" (UniqueName: \"kubernetes.io/projected/c752ba3a-30e7-4d5a-85f8-eea123b359aa-kube-api-access-tffzk\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826894 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-systemd\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-tmp-dir\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-iptables-alerter-script\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec13312c-4802-428d-bb87-0a70978eb388-dbus\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826979 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-sys-fs\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827026 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-cni-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827053 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-cni-binary-copy\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-var-lib-kubelet\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c562f70-0d3f-4bb9-8092-56fcdefbdb3c-konnectivity-ca\") pod \"konnectivity-agent-bb8gf\" (UID: \"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c\") " pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827128 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-tmp-dir\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-cnibin\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-env-overrides\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-host-slash\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.828542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827176 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-cnibin\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827211 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-host-slash\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-kubernetes\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827265 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-run\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0922e014-26df-43c7-93b0-dc3267e0c109-tmp\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-cni-binary-copy\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-conf-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-kubelet\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827350 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-var-lib-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-node-log\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-ovnkube-script-lib\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827414 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0922e014-26df-43c7-93b0-dc3267e0c109-etc-tuned\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827431 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827478 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-host\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827529 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-cni-multus\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827557 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-hostroot\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827564 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827581 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-log-socket\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.826838 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827707 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-socket-dir-parent\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827744 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-iptables-alerter-script\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-etc-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827782 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-device-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827821 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-hosts-file\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827874 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec13312c-4802-428d-bb87-0a70978eb388-dbus\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.827970 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.828028 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:31.328008059 +0000 UTC m=+3.090999921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828087 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-cni-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-ovnkube-config\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.827438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-system-cni-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.829950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-netns\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-run-ovn-kubernetes\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828299 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87w5l\" (UniqueName: \"kubernetes.io/projected/f6504eba-a3f1-416f-b1df-56308e89d459-kube-api-access-87w5l\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828317 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysctl-d\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-sys\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-daemon-config\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828364 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c24715a0-4a60-4813-b316-57aaa69e6247-ovn-node-metrics-cert\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828382 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828402 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-socket-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828434 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-registration-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-cni-bin\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-kubelet\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vxnw\" (UniqueName: \"kubernetes.io/projected/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-kube-api-access-4vxnw\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-ovn\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.830671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-cni-bin\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828536 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-system-cni-dir\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-etc-selinux\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828596 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-cni-binary-copy\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828650 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-etc-selinux\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828686 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-system-cni-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828709 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-cni-netd\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828731 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-run-netns\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828756 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-run-ovn-kubernetes\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828911 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c562f70-0d3f-4bb9-8092-56fcdefbdb3c-konnectivity-ca\") pod \"konnectivity-agent-bb8gf\" (UID: \"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c\") " pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828971 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-sysctl-d\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.828979 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-var-lib-kubelet\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829008 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-sys\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829038 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-var-lib-openvswitch\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829090 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-conf-dir\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.831286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829137 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-kubelet\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829165 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-cni-binary-copy\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829187 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-cni-bin\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829427 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-socket-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829443 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-multus-daemon-config\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829445 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/785527e0-6f25-4c91-88ba-a47aa06d9e94-registration-dir\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-run\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829500 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-run-ovn\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829509 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-host-cni-bin\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829549 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6504eba-a3f1-416f-b1df-56308e89d459-system-cni-dir\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829551 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0922e014-26df-43c7-93b0-dc3267e0c109-etc-kubernetes\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829585 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-env-overrides\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-host-var-lib-kubelet\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829805 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f6504eba-a3f1-416f-b1df-56308e89d459-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.829882 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c24715a0-4a60-4813-b316-57aaa69e6247-node-log\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.832134 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.830039 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c24715a0-4a60-4813-b316-57aaa69e6247-ovnkube-script-lib\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.833386 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.833361 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c24715a0-4a60-4813-b316-57aaa69e6247-ovn-node-metrics-cert\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.833958 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.833583 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c562f70-0d3f-4bb9-8092-56fcdefbdb3c-agent-certs\") pod \"konnectivity-agent-bb8gf\" (UID: \"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c\") " pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.834065 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.833965 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0922e014-26df-43c7-93b0-dc3267e0c109-tmp\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.834323 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.834303 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:56:30.834412 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.834331 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:56:30.834412 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.834346 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:30.834522 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:30.834416 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:31.334397172 +0000 UTC m=+3.097389014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:30.836362 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.835930 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0922e014-26df-43c7-93b0-dc3267e0c109-etc-tuned\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.837716 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.837357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5rn7\" (UniqueName: \"kubernetes.io/projected/c24715a0-4a60-4813-b316-57aaa69e6247-kube-api-access-k5rn7\") pod \"ovnkube-node-px95b\" (UID: \"c24715a0-4a60-4813-b316-57aaa69e6247\") " pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:30.840197 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.840173 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkh7x\" (UniqueName: \"kubernetes.io/projected/0922e014-26df-43c7-93b0-dc3267e0c109-kube-api-access-fkh7x\") pod \"tuned-4dtj9\" (UID: \"0922e014-26df-43c7-93b0-dc3267e0c109\") " pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:30.845419 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.845372 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhfxf\" (UniqueName: \"kubernetes.io/projected/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-kube-api-access-vhfxf\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:30.848901 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.848872 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87w5l\" (UniqueName: \"kubernetes.io/projected/f6504eba-a3f1-416f-b1df-56308e89d459-kube-api-access-87w5l\") pod \"multus-additional-cni-plugins-qkvzz\" (UID: \"f6504eba-a3f1-416f-b1df-56308e89d459\") " pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.850716 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.850685 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vxnw\" (UniqueName: \"kubernetes.io/projected/6aaa8e6e-cad3-4b39-af54-ea56b705ca77-kube-api-access-4vxnw\") pod \"multus-5bxrg\" (UID: \"6aaa8e6e-cad3-4b39-af54-ea56b705ca77\") " pod="openshift-multus/multus-5bxrg" Jul 08 16:56:30.851244 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.851196 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r96j4\" (UniqueName: \"kubernetes.io/projected/785527e0-6f25-4c91-88ba-a47aa06d9e94-kube-api-access-r96j4\") pod \"aws-ebs-csi-driver-node-wq8mf\" (UID: \"785527e0-6f25-4c91-88ba-a47aa06d9e94\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.851805 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.851760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tffzk\" (UniqueName: \"kubernetes.io/projected/c752ba3a-30e7-4d5a-85f8-eea123b359aa-kube-api-access-tffzk\") pod \"node-ca-7tb8p\" (UID: \"c752ba3a-30e7-4d5a-85f8-eea123b359aa\") " pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:30.852302 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.852279 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxfmj\" (UniqueName: \"kubernetes.io/projected/efd8aa5d-ccf1-4b87-9e54-a256d619c7e8-kube-api-access-mxfmj\") pod \"iptables-alerter-f78g2\" (UID: \"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8\") " pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.854667 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.854642 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f5r8\" (UniqueName: \"kubernetes.io/projected/6007f44b-deb5-47d6-a9ba-53ea03fe32e6-kube-api-access-5f5r8\") pod \"node-resolver-bc62n\" (UID: \"6007f44b-deb5-47d6-a9ba-53ea03fe32e6\") " pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.948247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.947981 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bc62n" Jul 08 16:56:30.955637 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.955418 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" Jul 08 16:56:30.959745 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:30.959712 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6007f44b_deb5_47d6_a9ba_53ea03fe32e6.slice/crio-04ff27a236ae514bde9a32643470f20e7ef0544422389ed4f8067610540d050e WatchSource:0}: Error finding container 04ff27a236ae514bde9a32643470f20e7ef0544422389ed4f8067610540d050e: Status 404 returned error can't find the container with id 04ff27a236ae514bde9a32643470f20e7ef0544422389ed4f8067610540d050e Jul 08 16:56:30.965945 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:30.965917 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6504eba_a3f1_416f_b1df_56308e89d459.slice/crio-ace0f1cd90addca8220c4cc2dfb80f57d5415673cc660e94aa9dea153fc163df WatchSource:0}: Error finding container ace0f1cd90addca8220c4cc2dfb80f57d5415673cc660e94aa9dea153fc163df: Status 404 returned error can't find the container with id ace0f1cd90addca8220c4cc2dfb80f57d5415673cc660e94aa9dea153fc163df Jul 08 16:56:30.969786 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.969761 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f78g2" Jul 08 16:56:30.977278 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.977013 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:30.981674 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:30.981651 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd8aa5d_ccf1_4b87_9e54_a256d619c7e8.slice/crio-ca7957335f102110cd76da4213ecdc632c3333c20363ffa2cefdc79407c67f9d WatchSource:0}: Error finding container ca7957335f102110cd76da4213ecdc632c3333c20363ffa2cefdc79407c67f9d: Status 404 returned error can't find the container with id ca7957335f102110cd76da4213ecdc632c3333c20363ffa2cefdc79407c67f9d Jul 08 16:56:30.984282 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.984260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" Jul 08 16:56:30.988254 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:30.988223 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c562f70_0d3f_4bb9_8092_56fcdefbdb3c.slice/crio-b8f8d784e4417760e097b30ce8bc9590ebf33b9b88c1c689b3820a976e1d3a05 WatchSource:0}: Error finding container b8f8d784e4417760e097b30ce8bc9590ebf33b9b88c1c689b3820a976e1d3a05: Status 404 returned error can't find the container with id b8f8d784e4417760e097b30ce8bc9590ebf33b9b88c1c689b3820a976e1d3a05 Jul 08 16:56:30.995858 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:30.995809 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785527e0_6f25_4c91_88ba_a47aa06d9e94.slice/crio-f837740ad432fb5255c10b917c99b471d203eda410a0c201945831ed17064a72 WatchSource:0}: Error finding container f837740ad432fb5255c10b917c99b471d203eda410a0c201945831ed17064a72: Status 404 returned error can't find the container with id f837740ad432fb5255c10b917c99b471d203eda410a0c201945831ed17064a72 Jul 08 16:56:30.996829 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:30.996598 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" Jul 08 16:56:31.004382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.004195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5bxrg" Jul 08 16:56:31.008371 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.008026 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:31.013276 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.013256 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7tb8p" Jul 08 16:56:31.014397 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:31.014370 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0922e014_26df_43c7_93b0_dc3267e0c109.slice/crio-f89d079246533ccf0e114ac9e7a5c21f6e152389e9e6166485f1768c96c99757 WatchSource:0}: Error finding container f89d079246533ccf0e114ac9e7a5c21f6e152389e9e6166485f1768c96c99757: Status 404 returned error can't find the container with id f89d079246533ccf0e114ac9e7a5c21f6e152389e9e6166485f1768c96c99757 Jul 08 16:56:31.020902 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:31.020871 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24715a0_4a60_4813_b316_57aaa69e6247.slice/crio-fcddcd01f9190f88bfc7ff4610f4383fb90e2ec5831a1172aeb66caad67c2e11 WatchSource:0}: Error finding container fcddcd01f9190f88bfc7ff4610f4383fb90e2ec5831a1172aeb66caad67c2e11: Status 404 returned error can't find the container with id fcddcd01f9190f88bfc7ff4610f4383fb90e2ec5831a1172aeb66caad67c2e11 Jul 08 16:56:31.022420 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:31.022336 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aaa8e6e_cad3_4b39_af54_ea56b705ca77.slice/crio-c350e383b6e0fbb89386a5f5784cfc1de70607aadb174346a1200af6e49ac7d6 WatchSource:0}: Error finding container c350e383b6e0fbb89386a5f5784cfc1de70607aadb174346a1200af6e49ac7d6: Status 404 returned error can't find the container with id c350e383b6e0fbb89386a5f5784cfc1de70607aadb174346a1200af6e49ac7d6 Jul 08 16:56:31.034801 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:56:31.034768 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc752ba3a_30e7_4d5a_85f8_eea123b359aa.slice/crio-aabc9683776298a6aee586c53e420435ec5d399fdb02be4f234f09c2ecaa53ca WatchSource:0}: Error finding container aabc9683776298a6aee586c53e420435ec5d399fdb02be4f234f09c2ecaa53ca: Status 404 returned error can't find the container with id aabc9683776298a6aee586c53e420435ec5d399fdb02be4f234f09c2ecaa53ca Jul 08 16:56:31.335302 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.333577 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:31.335302 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.333689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:31.335302 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.333839 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:31.335302 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.333926 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:32.333908975 +0000 UTC m=+4.096900819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:31.335302 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.333840 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:31.335302 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.334369 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:32.334355474 +0000 UTC m=+4.097347315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:31.434130 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.434082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:31.434315 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.434295 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:56:31.434397 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.434328 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:56:31.434397 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.434344 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:31.434509 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.434420 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:32.434398324 +0000 UTC m=+4.197390165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:31.685731 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.685607 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:51:30 +0000 UTC" deadline="2028-03-23 21:40:53.02686111 +0000 UTC" Jul 08 16:56:31.685731 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.685673 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14980h44m21.341190425s" Jul 08 16:56:31.790787 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.790744 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:31.790965 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:31.790910 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:31.828558 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.828510 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5bxrg" event={"ID":"6aaa8e6e-cad3-4b39-af54-ea56b705ca77","Type":"ContainerStarted","Data":"c350e383b6e0fbb89386a5f5784cfc1de70607aadb174346a1200af6e49ac7d6"} Jul 08 16:56:31.830001 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.829940 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" event={"ID":"0922e014-26df-43c7-93b0-dc3267e0c109","Type":"ContainerStarted","Data":"f89d079246533ccf0e114ac9e7a5c21f6e152389e9e6166485f1768c96c99757"} Jul 08 16:56:31.831274 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.831244 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" event={"ID":"785527e0-6f25-4c91-88ba-a47aa06d9e94","Type":"ContainerStarted","Data":"f837740ad432fb5255c10b917c99b471d203eda410a0c201945831ed17064a72"} Jul 08 16:56:31.832503 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.832478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bb8gf" event={"ID":"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c","Type":"ContainerStarted","Data":"b8f8d784e4417760e097b30ce8bc9590ebf33b9b88c1c689b3820a976e1d3a05"} Jul 08 16:56:31.833633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.833601 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f78g2" event={"ID":"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8","Type":"ContainerStarted","Data":"ca7957335f102110cd76da4213ecdc632c3333c20363ffa2cefdc79407c67f9d"} Jul 08 16:56:31.834903 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.834882 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerStarted","Data":"ace0f1cd90addca8220c4cc2dfb80f57d5415673cc660e94aa9dea153fc163df"} Jul 08 16:56:31.835983 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.835963 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7tb8p" event={"ID":"c752ba3a-30e7-4d5a-85f8-eea123b359aa","Type":"ContainerStarted","Data":"aabc9683776298a6aee586c53e420435ec5d399fdb02be4f234f09c2ecaa53ca"} Jul 08 16:56:31.837040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.837021 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"fcddcd01f9190f88bfc7ff4610f4383fb90e2ec5831a1172aeb66caad67c2e11"} Jul 08 16:56:31.838152 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:31.838118 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bc62n" event={"ID":"6007f44b-deb5-47d6-a9ba-53ea03fe32e6","Type":"ContainerStarted","Data":"04ff27a236ae514bde9a32643470f20e7ef0544422389ed4f8067610540d050e"} Jul 08 16:56:32.341728 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.341682 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:32.341933 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.341767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:32.341933 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.341911 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:32.342059 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.341980 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:34.341957359 +0000 UTC m=+6.104949222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:32.342424 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.342406 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:32.342528 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.342459 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:34.342443924 +0000 UTC m=+6.105435769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:32.442219 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.442177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:32.442395 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.442356 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:56:32.442395 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.442375 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:56:32.442395 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.442389 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:32.442532 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.442449 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:34.442431145 +0000 UTC m=+6.205422986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:32.686841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.686624 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:51:30 +0000 UTC" deadline="2028-04-09 05:59:02.188619022 +0000 UTC" Jul 08 16:56:32.686841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.686840 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15373h2m29.501784902s" Jul 08 16:56:32.791403 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.790766 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:32.791403 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.790890 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:32.791403 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.790950 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:32.791403 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:32.791060 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:32.856351 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:32.856234 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" event={"ID":"0052804689573a175003f01e231a790e","Type":"ContainerStarted","Data":"e82ad2a47e2bb44ca724b54e323d751d4e1777765d140ccc22a4679a5ebd8292"} Jul 08 16:56:33.790871 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:33.790836 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:33.791048 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:33.790994 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:33.867553 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:33.866434 2563 generic.go:356] "Generic (PLEG): container finished" podID="ec3d18bceb905803cc4190ce836dffb9" containerID="5ce3e021fb63cd7d35dedd4049664859b447dc51bd138ee63898abdfe80e5050" exitCode=0 Jul 08 16:56:33.867553 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:33.867369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" event={"ID":"ec3d18bceb905803cc4190ce836dffb9","Type":"ContainerDied","Data":"5ce3e021fb63cd7d35dedd4049664859b447dc51bd138ee63898abdfe80e5050"} Jul 08 16:56:33.889140 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:33.889080 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-156.ec2.internal" podStartSLOduration=4.889059178 podStartE2EDuration="4.889059178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:32.877432581 +0000 UTC m=+4.640424439" watchObservedRunningTime="2026-07-08 16:56:33.889059178 +0000 UTC m=+5.652051038" Jul 08 16:56:34.356456 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:34.356422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:34.356662 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:34.356499 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:34.356662 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.356623 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:34.356791 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.356684 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:38.356666136 +0000 UTC m=+10.119657977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:34.357084 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.357063 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:34.357208 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.357122 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:38.357107363 +0000 UTC m=+10.120099214 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:34.457579 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:34.457540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:34.457802 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.457757 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:56:34.457802 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.457783 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:56:34.457802 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.457796 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:34.457965 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.457879 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:38.4578618 +0000 UTC m=+10.220853638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:34.793762 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:34.793641 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:34.793924 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.793800 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:34.795443 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:34.795420 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:34.795572 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:34.795551 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:35.791608 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:35.790549 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:35.791608 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:35.790688 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:36.790463 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:36.790389 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:36.790591 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:36.790517 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:36.790591 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:36.790582 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:36.790726 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:36.790692 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:37.791221 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:37.791190 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:37.791734 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:37.791324 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:38.392984 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:38.392944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:38.393140 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:38.393018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:38.393214 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.393174 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:38.393263 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.393237 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:46.393217237 +0000 UTC m=+18.156209080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:38.393263 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.393175 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:38.393364 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.393309 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:46.393291678 +0000 UTC m=+18.156283521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:38.493963 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:38.493922 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:38.494177 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.494157 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:56:38.494259 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.494189 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:56:38.494259 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.494203 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:38.494358 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.494270 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:46.494251994 +0000 UTC m=+18.257243835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:38.792299 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:38.791654 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:38.792299 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.791758 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:38.792299 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:38.792107 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:38.792299 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:38.792205 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:39.791161 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:39.791111 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:39.791322 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:39.791269 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:40.791078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:40.791039 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:40.791548 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:40.791194 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:40.791548 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:40.791054 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:40.791663 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:40.791627 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:41.790453 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:41.790411 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:41.790638 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:41.790541 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:42.791013 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:42.790935 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:42.791453 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:42.791055 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:42.791453 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:42.791107 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:42.791453 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:42.791220 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:43.790564 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:43.790527 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:43.790744 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:43.790648 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:44.790365 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:44.790334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:44.790797 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:44.790460 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:44.790797 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:44.790526 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:44.790797 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:44.790674 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:45.790565 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:45.790533 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:45.791023 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:45.790647 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:46.459154 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:46.459043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:46.459154 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:46.459118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:46.459374 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.459231 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:46.459374 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.459233 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:46.459374 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.459293 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:02.459277382 +0000 UTC m=+34.222269222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:56:46.459374 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.459309 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:02.459301842 +0000 UTC m=+34.222293681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:56:46.560411 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:46.560374 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:46.560592 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.560569 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:56:46.560651 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.560600 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:56:46.560651 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.560616 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:46.560715 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.560681 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:02.560662726 +0000 UTC m=+34.323654571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:56:46.791179 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:46.791080 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:46.791179 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:46.791100 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:46.791622 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.791258 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:46.791622 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:46.791394 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:47.790450 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:47.790417 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:47.790649 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:47.790539 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:48.791011 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:48.790976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:48.791477 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:48.791041 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:48.791477 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:48.791126 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:48.791477 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:48.791289 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:49.790412 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:49.790373 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:49.790585 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:49.790501 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:50.790723 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.790496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:50.791078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.790555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:50.791078 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:50.790839 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:50.791078 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:50.790917 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:50.901267 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.901178 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5bxrg" event={"ID":"6aaa8e6e-cad3-4b39-af54-ea56b705ca77","Type":"ContainerStarted","Data":"7d77a138696473430723218101b3daf4a76c63075720555a1989bd80227df858"} Jul 08 16:56:50.902663 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.902640 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" event={"ID":"0922e014-26df-43c7-93b0-dc3267e0c109","Type":"ContainerStarted","Data":"c5de2972b0a36edf4e52329cd740ac30106da28965d9036d79a721f3dc29371a"} Jul 08 16:56:50.904038 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.904015 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" event={"ID":"785527e0-6f25-4c91-88ba-a47aa06d9e94","Type":"ContainerStarted","Data":"5650fbcfc7d56035d98ef50bb690cf084e874dd4a22254b07960f40d3f7233d3"} Jul 08 16:56:50.905511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.905485 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bb8gf" event={"ID":"1c562f70-0d3f-4bb9-8092-56fcdefbdb3c","Type":"ContainerStarted","Data":"57664e9569284f100cbd7b6d7dbbacec3b40a09b0a64772c9e074ec4b2af3580"} Jul 08 16:56:50.906970 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.906947 2563 generic.go:356] "Generic (PLEG): container finished" podID="f6504eba-a3f1-416f-b1df-56308e89d459" containerID="b736f5d25ebfa2fe2a6bfbfc16e446587cc97fa58904d1bd18072bfd995e9bdd" exitCode=0 Jul 08 16:56:50.907057 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.907020 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerDied","Data":"b736f5d25ebfa2fe2a6bfbfc16e446587cc97fa58904d1bd18072bfd995e9bdd"} Jul 08 16:56:50.909014 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.908990 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" event={"ID":"ec3d18bceb905803cc4190ce836dffb9","Type":"ContainerStarted","Data":"c1440c7baf4e59cd5a78c4b72cd1c3aba8b0130d3dfb9622fa1524a97d2483ee"} Jul 08 16:56:50.910490 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.910469 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7tb8p" event={"ID":"c752ba3a-30e7-4d5a-85f8-eea123b359aa","Type":"ContainerStarted","Data":"1546760de49cda82f39c6bce17c520f16f99bfc4f15180d0f44e0f22ad99667e"} Jul 08 16:56:50.911922 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.911905 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bc62n" event={"ID":"6007f44b-deb5-47d6-a9ba-53ea03fe32e6","Type":"ContainerStarted","Data":"638935ec7e3d3b3ba77f5be6e1907e8390dea3c3a5e6577073a72e8b4eff575d"} Jul 08 16:56:50.924288 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.924247 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5bxrg" podStartSLOduration=2.500874666 podStartE2EDuration="21.924234706s" totalImagesPullingTime="19.42336004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:50.922945007 +0000 UTC m=+22.685936868" watchObservedRunningTime="2026-07-08 16:56:50.924234706 +0000 UTC m=+22.687226566" Jul 08 16:56:50.959946 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.959895 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-156.ec2.internal" podStartSLOduration=20.959879813 podStartE2EDuration="20.959879813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:50.959701911 +0000 UTC m=+22.722693771" watchObservedRunningTime="2026-07-08 16:56:50.959879813 +0000 UTC m=+22.722871656" Jul 08 16:56:50.960591 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.960561 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7tb8p" podStartSLOduration=6.919365871 podStartE2EDuration="21.960551901s" totalImagesPullingTime="15.04118603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:50.943825479 +0000 UTC m=+22.706817340" watchObservedRunningTime="2026-07-08 16:56:50.960551901 +0000 UTC m=+22.723543761" Jul 08 16:56:50.985100 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:50.985049 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4dtj9" podStartSLOduration=2.5709154610000002 podStartE2EDuration="21.985034587s" totalImagesPullingTime="19.414119126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:50.984744664 +0000 UTC m=+22.747736547" watchObservedRunningTime="2026-07-08 16:56:50.985034587 +0000 UTC m=+22.748026446" Jul 08 16:56:51.007937 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.007784 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bc62n" podStartSLOduration=2.639893956 podStartE2EDuration="22.007764752s" totalImagesPullingTime="19.367870796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:51.007185594 +0000 UTC m=+22.770177449" watchObservedRunningTime="2026-07-08 16:56:51.007764752 +0000 UTC m=+22.770756613" Jul 08 16:56:51.050897 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.050849 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bb8gf" podStartSLOduration=2.713162988 podStartE2EDuration="22.050833869s" totalImagesPullingTime="19.337670881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:51.050341773 +0000 UTC m=+22.813333632" watchObservedRunningTime="2026-07-08 16:56:51.050833869 +0000 UTC m=+22.813825729" Jul 08 16:56:51.791062 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.790874 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:51.791716 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:51.791167 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:51.917575 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.917537 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f78g2" event={"ID":"efd8aa5d-ccf1-4b87-9e54-a256d619c7e8","Type":"ContainerStarted","Data":"b907f3197ce425e0ca500d53f74ca0360c3fde25e373676e2b6b3746321871df"} Jul 08 16:56:51.922781 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.922746 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"5690e28a04b008c6653f7e1fd42e168f446640028eb0784894cfaafe389eb09e"} Jul 08 16:56:51.922932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.922817 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"207535d829aef78ee3044a3fea8be25b8978ac650db102f29454d45bc61ae6cc"} Jul 08 16:56:51.922932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.922879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"1f61d437e1bbc61c97dcf513b5bf697632c1ca95a3c5dc5d8a372ccfcea3db68"} Jul 08 16:56:51.922932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.922927 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"9129d27f3af67cbe755a6bc72626e11ac85fe3abc31170c82e7d00191ab9b56f"} Jul 08 16:56:51.923141 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.922974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"7e73541310ca35aaaef29111b78e130bc973ba1c086e1e9b6fbaf34de0d1f13d"} Jul 08 16:56:51.923141 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.923025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"d3170c68aae7a40c097e0937686ac29b2c136c57b01f8c5e4e9483ffd029e5f8"} Jul 08 16:56:51.936302 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:51.936191 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f78g2" podStartSLOduration=3.4714463159999998 podStartE2EDuration="22.936175868s" totalImagesPullingTime="19.464729552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:51.93552776 +0000 UTC m=+23.698519611" watchObservedRunningTime="2026-07-08 16:56:51.936175868 +0000 UTC m=+23.699167730" Jul 08 16:56:52.395894 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.395847 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 16:56:52.685743 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.685606 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T16:56:52.395865003Z","UUID":"83dbcd1d-8c6e-48d8-9dab-ccd7181a116d","Handler":null,"Name":"","Endpoint":""} Jul 08 16:56:52.687293 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.687275 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 16:56:52.687423 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.687306 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 16:56:52.790974 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.790944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:52.791257 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.790953 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:52.791257 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:52.791064 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:52.791257 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:52.791160 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:52.927794 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:52.927754 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" event={"ID":"785527e0-6f25-4c91-88ba-a47aa06d9e94","Type":"ContainerStarted","Data":"4e36cbf75c96fb11877dbc3ea434eb59ba1c29f862a87bc651da9a3c282edf18"} Jul 08 16:56:53.790694 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:53.790660 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:53.790945 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:53.790775 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:54.411960 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:54.411925 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:54.412689 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:54.412662 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:54.790654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:54.790528 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:54.790805 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:54.790663 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:54.790805 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:54.790719 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:54.790898 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:54.790830 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:54.932655 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:54.932625 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:54.933219 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:54.933198 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bb8gf" Jul 08 16:56:55.791031 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.790953 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:55.791631 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:55.791077 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:55.936193 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.936163 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" event={"ID":"785527e0-6f25-4c91-88ba-a47aa06d9e94","Type":"ContainerStarted","Data":"f979c580bcb147fbc02303ef90352deeae1c65c50e040485c3e34be572b46db1"} Jul 08 16:56:55.938003 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.937979 2563 generic.go:356] "Generic (PLEG): container finished" podID="f6504eba-a3f1-416f-b1df-56308e89d459" containerID="a541a7e6ad06826753814f720b7ac521b19c38743e7a18d8ef025092cff61914" exitCode=0 Jul 08 16:56:55.938118 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.938052 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerDied","Data":"a541a7e6ad06826753814f720b7ac521b19c38743e7a18d8ef025092cff61914"} Jul 08 16:56:55.944900 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.944870 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"9f148759181a22c0d1362f46cd7ed52b9a8e6ccd33c56711d3436ff9d182f022"} Jul 08 16:56:55.951006 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.950979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:56:55.996735 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:55.996685 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wq8mf" podStartSLOduration=3.175685358 podStartE2EDuration="26.996669257s" totalImagesPullingTime="23.820983899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:55.967544156 +0000 UTC m=+27.730536016" watchObservedRunningTime="2026-07-08 16:56:55.996669257 +0000 UTC m=+27.759661165" Jul 08 16:56:56.073589 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:56.073507 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:56:56.790887 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:56.790817 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:56.791052 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:56.790920 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:56.791052 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:56.790977 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:56.791360 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:56.791090 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:57.791378 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.791139 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:57.791832 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:57.791412 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:57.950854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.950821 2563 generic.go:356] "Generic (PLEG): container finished" podID="f6504eba-a3f1-416f-b1df-56308e89d459" containerID="719fa20d115909427d16ea38a71577a75fceed6113f7be73ed62053788e57ec6" exitCode=0 Jul 08 16:56:57.951027 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.950910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerDied","Data":"719fa20d115909427d16ea38a71577a75fceed6113f7be73ed62053788e57ec6"} Jul 08 16:56:57.954445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.954425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" event={"ID":"c24715a0-4a60-4813-b316-57aaa69e6247","Type":"ContainerStarted","Data":"bed5c77978fb400712e2701d6a3be6040f1b8fafa80d10d12a6271ebb9ee6906"} Jul 08 16:56:57.954626 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.954610 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:57.954674 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.954662 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:57.970192 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:57.970171 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:58.011905 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.011855 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" podStartSLOduration=8.915727059 podStartE2EDuration="29.011840694s" totalImagesPullingTime="20.096113635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:56:58.011522204 +0000 UTC m=+29.774514066" watchObservedRunningTime="2026-07-08 16:56:58.011840694 +0000 UTC m=+29.774832553" Jul 08 16:56:58.230607 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.230521 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:56:58.335585 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.335553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:56:58.791032 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.791000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:58.791235 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:58.791100 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:58.791235 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.791153 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:58.791348 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:58.791237 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:58.958083 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.958054 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:58.976106 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:58.976076 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:56:59.791499 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.791272 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:59.791837 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:59.791807 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:59.806800 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.806769 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nkwpj"] Jul 08 16:56:59.813287 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.813264 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pb29q"] Jul 08 16:56:59.813411 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.813377 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:56:59.813501 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:59.813481 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:56:59.828847 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.828775 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lgbxp"] Jul 08 16:56:59.828954 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.828874 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:56:59.828995 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:59.828951 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:56:59.961841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.961809 2563 generic.go:356] "Generic (PLEG): container finished" podID="f6504eba-a3f1-416f-b1df-56308e89d459" containerID="94d68d839c3f2a5a366c683b2c6c16355402b7aac4e88b180a5eb8d92f1c786f" exitCode=0 Jul 08 16:56:59.962323 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.961878 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerDied","Data":"94d68d839c3f2a5a366c683b2c6c16355402b7aac4e88b180a5eb8d92f1c786f"} Jul 08 16:56:59.962323 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.961940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:56:59.962323 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:56:59.962044 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:56:59.964232 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:56:59.964211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:00.068216 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:00.068188 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:00.775602 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:00.775568 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:00.885291 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:00.885263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:01.790563 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:01.790478 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:01.791059 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:01.790585 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:01.791059 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:01.790591 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:57:01.791059 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:01.790636 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:01.791059 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:01.790749 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:57:01.791059 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:01.790835 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:57:02.480122 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:02.480043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:02.480122 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:02.480092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:02.480322 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.480210 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:57:02.480322 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.480223 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 16:57:02.480322 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.480272 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs podName:747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:34.480257175 +0000 UTC m=+66.243249018 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs") pod "network-metrics-daemon-nkwpj" (UID: "747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 16:57:02.480322 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.480286 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret podName:ec13312c-4802-428d-bb87-0a70978eb388 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:34.480280462 +0000 UTC m=+66.243272299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret") pod "global-pull-secret-syncer-lgbxp" (UID: "ec13312c-4802-428d-bb87-0a70978eb388") : object "kube-system"/"original-pull-secret" not registered Jul 08 16:57:02.581440 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:02.581407 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:02.581625 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.581545 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 16:57:02.581625 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.581563 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 16:57:02.581625 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.581572 2563 projected.go:196] Error preparing data for projected volume kube-api-access-cqj5d for pod openshift-network-diagnostics/network-check-target-pb29q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:57:02.581775 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:02.581650 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d podName:12a12eb7-e5a5-42f0-9a59-0c70f6e87268 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:34.58163169 +0000 UTC m=+66.344623546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqj5d" (UniqueName: "kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d") pod "network-check-target-pb29q" (UID: "12a12eb7-e5a5-42f0-9a59-0c70f6e87268") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 16:57:03.790947 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:03.790912 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:03.790947 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:03.790940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:03.791511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:03.790939 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:03.791511 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:03.791023 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgbxp" podUID="ec13312c-4802-428d-bb87-0a70978eb388" Jul 08 16:57:03.791511 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:03.791110 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nkwpj" podUID="747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687" Jul 08 16:57:03.791511 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:03.791191 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pb29q" podUID="12a12eb7-e5a5-42f0-9a59-0c70f6e87268" Jul 08 16:57:04.306660 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.306579 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:04.428803 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.428773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:04.566214 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.566187 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-156.ec2.internal" event="NodeReady" Jul 08 16:57:04.566376 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.566351 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 16:57:04.638766 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.638729 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-brkmd"] Jul 08 16:57:04.656354 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.656323 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x7crb"] Jul 08 16:57:04.656510 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.656435 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.670428 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.670405 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.673332 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.673293 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 16:57:04.674408 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.674386 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-brkmd"] Jul 08 16:57:04.674575 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.674560 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9p4zl\"" Jul 08 16:57:04.674850 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.674835 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 16:57:04.675923 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.675906 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 16:57:04.676965 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.676943 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 16:57:04.677048 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.677024 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-44wwq\"" Jul 08 16:57:04.677944 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.677930 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 16:57:04.685721 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.685703 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x7crb"] Jul 08 16:57:04.800507 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.800478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.801093 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.800511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxlvr\" (UniqueName: \"kubernetes.io/projected/3e40726d-a930-4435-9ba2-d7ede73e48a8-kube-api-access-vxlvr\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.801093 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.800536 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnh7f\" (UniqueName: \"kubernetes.io/projected/dd0ce6a4-e00e-4f12-aba0-5524ccbac037-kube-api-access-pnh7f\") pod \"ingress-canary-brkmd\" (UID: \"dd0ce6a4-e00e-4f12-aba0-5524ccbac037\") " pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.801093 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.800627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd0ce6a4-e00e-4f12-aba0-5524ccbac037-cert\") pod \"ingress-canary-brkmd\" (UID: \"dd0ce6a4-e00e-4f12-aba0-5524ccbac037\") " pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.801093 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.800671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e40726d-a930-4435-9ba2-d7ede73e48a8-tmp-dir\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.801093 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.800705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3e40726d-a930-4435-9ba2-d7ede73e48a8-config-volume\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.901509 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnh7f\" (UniqueName: \"kubernetes.io/projected/dd0ce6a4-e00e-4f12-aba0-5524ccbac037-kube-api-access-pnh7f\") pod \"ingress-canary-brkmd\" (UID: \"dd0ce6a4-e00e-4f12-aba0-5524ccbac037\") " pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.901509 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd0ce6a4-e00e-4f12-aba0-5524ccbac037-cert\") pod \"ingress-canary-brkmd\" (UID: \"dd0ce6a4-e00e-4f12-aba0-5524ccbac037\") " pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.901509 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901503 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e40726d-a930-4435-9ba2-d7ede73e48a8-tmp-dir\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.901774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3e40726d-a930-4435-9ba2-d7ede73e48a8-config-volume\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.901774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901548 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.901774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxlvr\" (UniqueName: \"kubernetes.io/projected/3e40726d-a930-4435-9ba2-d7ede73e48a8-kube-api-access-vxlvr\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.901774 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:04.901690 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 16:57:04.901774 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:04.901760 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls podName:3e40726d-a930-4435-9ba2-d7ede73e48a8 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:05.401739656 +0000 UTC m=+37.164731508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls") pod "dns-default-x7crb" (UID: "3e40726d-a930-4435-9ba2-d7ede73e48a8") : secret "dns-default-metrics-tls" not found Jul 08 16:57:04.901994 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.901859 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e40726d-a930-4435-9ba2-d7ede73e48a8-tmp-dir\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.902107 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.902088 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3e40726d-a930-4435-9ba2-d7ede73e48a8-config-volume\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.905358 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.905332 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd0ce6a4-e00e-4f12-aba0-5524ccbac037-cert\") pod \"ingress-canary-brkmd\" (UID: \"dd0ce6a4-e00e-4f12-aba0-5524ccbac037\") " pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.915402 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.915368 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxlvr\" (UniqueName: \"kubernetes.io/projected/3e40726d-a930-4435-9ba2-d7ede73e48a8-kube-api-access-vxlvr\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:04.915515 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.915495 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnh7f\" (UniqueName: \"kubernetes.io/projected/dd0ce6a4-e00e-4f12-aba0-5524ccbac037-kube-api-access-pnh7f\") pod \"ingress-canary-brkmd\" (UID: \"dd0ce6a4-e00e-4f12-aba0-5524ccbac037\") " pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.964405 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.964364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-brkmd" Jul 08 16:57:04.975696 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.975663 2563 generic.go:356] "Generic (PLEG): container finished" podID="f6504eba-a3f1-416f-b1df-56308e89d459" containerID="f90c28510923a81c10b429624024101bbb424947b8964a931f74ce993b1a9315" exitCode=0 Jul 08 16:57:04.975812 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:04.975699 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerDied","Data":"f90c28510923a81c10b429624024101bbb424947b8964a931f74ce993b1a9315"} Jul 08 16:57:05.142350 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.142186 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-brkmd"] Jul 08 16:57:05.159976 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:05.159895 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0ce6a4_e00e_4f12_aba0_5524ccbac037.slice/crio-d34099f9555563051a5f306af089fd024bdfc49ba2b44d6755d556fcc92f9366 WatchSource:0}: Error finding container d34099f9555563051a5f306af089fd024bdfc49ba2b44d6755d556fcc92f9366: Status 404 returned error can't find the container with id d34099f9555563051a5f306af089fd024bdfc49ba2b44d6755d556fcc92f9366 Jul 08 16:57:05.404806 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.404767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:05.404971 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:05.404886 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 16:57:05.404971 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:05.404938 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls podName:3e40726d-a930-4435-9ba2-d7ede73e48a8 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:06.404924867 +0000 UTC m=+38.167916705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls") pod "dns-default-x7crb" (UID: "3e40726d-a930-4435-9ba2-d7ede73e48a8") : secret "dns-default-metrics-tls" not found Jul 08 16:57:05.790837 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.790803 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:05.791035 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.790806 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:05.791103 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.790817 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:05.794307 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.794284 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-594z6\"" Jul 08 16:57:05.794908 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.794885 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 16:57:05.795006 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.794910 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pgff9\"" Jul 08 16:57:05.795006 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.794916 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 16:57:05.795006 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.794919 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 16:57:05.795204 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.795191 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 16:57:05.981524 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.981487 2563 generic.go:356] "Generic (PLEG): container finished" podID="f6504eba-a3f1-416f-b1df-56308e89d459" containerID="2ecb124776bbf40c4ea0793de52d6e10d585fd1023de148bc3815df9afd887cf" exitCode=0 Jul 08 16:57:05.981978 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.981574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerDied","Data":"2ecb124776bbf40c4ea0793de52d6e10d585fd1023de148bc3815df9afd887cf"} Jul 08 16:57:05.982833 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:05.982798 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-brkmd" event={"ID":"dd0ce6a4-e00e-4f12-aba0-5524ccbac037","Type":"ContainerStarted","Data":"d34099f9555563051a5f306af089fd024bdfc49ba2b44d6755d556fcc92f9366"} Jul 08 16:57:06.413636 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.413549 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:06.413770 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:06.413685 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 16:57:06.413770 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:06.413755 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls podName:3e40726d-a930-4435-9ba2-d7ede73e48a8 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:08.413733263 +0000 UTC m=+40.176725102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls") pod "dns-default-x7crb" (UID: "3e40726d-a930-4435-9ba2-d7ede73e48a8") : secret "dns-default-metrics-tls" not found Jul 08 16:57:06.524448 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.524414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:06.560059 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.560032 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr"] Jul 08 16:57:06.562157 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.562123 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" Jul 08 16:57:06.565553 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.565530 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-h7rvj\"" Jul 08 16:57:06.565862 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.565849 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 16:57:06.565862 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.565853 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 16:57:06.576613 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.576592 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr"] Jul 08 16:57:06.650368 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.650336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:06.716531 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.716341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bc5b\" (UniqueName: \"kubernetes.io/projected/03e9c757-1324-4ebc-be00-bce7536d7986-kube-api-access-4bc5b\") pod \"migrator-d57d58cc9-vgwjr\" (UID: \"03e9c757-1324-4ebc-be00-bce7536d7986\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" Jul 08 16:57:06.749974 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.749920 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:06.817284 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.817248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bc5b\" (UniqueName: \"kubernetes.io/projected/03e9c757-1324-4ebc-be00-bce7536d7986-kube-api-access-4bc5b\") pod \"migrator-d57d58cc9-vgwjr\" (UID: \"03e9c757-1324-4ebc-be00-bce7536d7986\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" Jul 08 16:57:06.832802 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.832773 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bc5b\" (UniqueName: \"kubernetes.io/projected/03e9c757-1324-4ebc-be00-bce7536d7986-kube-api-access-4bc5b\") pod \"migrator-d57d58cc9-vgwjr\" (UID: \"03e9c757-1324-4ebc-be00-bce7536d7986\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" Jul 08 16:57:06.871664 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.871631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" Jul 08 16:57:06.989446 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:06.989369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" event={"ID":"f6504eba-a3f1-416f-b1df-56308e89d459","Type":"ContainerStarted","Data":"7d5abac1e079238da6a2e453e853286f005a13b8c43ad7cece27249a5f859f68"} Jul 08 16:57:07.028832 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.028767 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qkvzz" podStartSLOduration=7.563553189 podStartE2EDuration="38.028752185s" totalImagesPullingTime="30.465198996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:07.026963915 +0000 UTC m=+38.789955777" watchObservedRunningTime="2026-07-08 16:57:07.028752185 +0000 UTC m=+38.791744045" Jul 08 16:57:07.165642 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.165604 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.272020 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.271930 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.354426 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.354389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.523740 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.523665 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.538304 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.538274 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr"] Jul 08 16:57:07.544351 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:07.544329 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e9c757_1324_4ebc_be00_bce7536d7986.slice/crio-6e8e7a76c4a0128e69a096c6da8e512572e272a3a75992b40aded3312db66e74 WatchSource:0}: Error finding container 6e8e7a76c4a0128e69a096c6da8e512572e272a3a75992b40aded3312db66e74: Status 404 returned error can't find the container with id 6e8e7a76c4a0128e69a096c6da8e512572e272a3a75992b40aded3312db66e74 Jul 08 16:57:07.546359 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.546318 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zqtkk"] Jul 08 16:57:07.548665 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.548648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.553365 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.553324 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 16:57:07.554256 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.554239 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 16:57:07.554256 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.554251 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 16:57:07.554393 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.554259 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 16:57:07.554393 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.554242 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j5hhq\"" Jul 08 16:57:07.568723 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.568701 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqtkk"] Jul 08 16:57:07.626671 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.626640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.724105 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.724064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.724105 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.724112 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.724370 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.724229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrsxv\" (UniqueName: \"kubernetes.io/projected/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-api-access-jrsxv\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.724370 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.724262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fecbdd8-5abd-4b47-9e87-2957eda23b79-crio-socket\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.724370 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.724307 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fecbdd8-5abd-4b47-9e87-2957eda23b79-data-volume\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.825635 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.825547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrsxv\" (UniqueName: \"kubernetes.io/projected/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-api-access-jrsxv\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.825635 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.825594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fecbdd8-5abd-4b47-9e87-2957eda23b79-crio-socket\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.825850 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.825637 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fecbdd8-5abd-4b47-9e87-2957eda23b79-data-volume\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.825850 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.825684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.825850 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:07.825757 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 16:57:07.825850 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.825754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.825850 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:07.825820 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:08.325801544 +0000 UTC m=+40.088793401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : configmap "kube-rbac-proxy" not found Jul 08 16:57:07.825850 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:07.825832 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 16:57:07.826122 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.825861 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8fecbdd8-5abd-4b47-9e87-2957eda23b79-crio-socket\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.826122 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:07.825890 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:08.325871968 +0000 UTC m=+40.088863808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : secret "insights-runtime-extractor-tls" not found Jul 08 16:57:07.826122 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.826008 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fecbdd8-5abd-4b47-9e87-2957eda23b79-data-volume\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.851050 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.851025 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrsxv\" (UniqueName: \"kubernetes.io/projected/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-api-access-jrsxv\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:07.880834 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.880797 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.973158 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.973113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:07.992900 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.992863 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" event={"ID":"03e9c757-1324-4ebc-be00-bce7536d7986","Type":"ContainerStarted","Data":"6e8e7a76c4a0128e69a096c6da8e512572e272a3a75992b40aded3312db66e74"} Jul 08 16:57:07.994189 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:07.994164 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-brkmd" event={"ID":"dd0ce6a4-e00e-4f12-aba0-5524ccbac037","Type":"ContainerStarted","Data":"fd85f5cd336ac69419a0394cffa172ac3b8123e679980fcf12e1d6ba431a8f09"} Jul 08 16:57:08.014734 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.014688 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-brkmd" podStartSLOduration=1.7413238469999999 podStartE2EDuration="4.014671941s" totalImagesPullingTime="2.273348094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:08.013620682 +0000 UTC m=+39.776612557" watchObservedRunningTime="2026-07-08 16:57:08.014671941 +0000 UTC m=+39.777663802" Jul 08 16:57:08.317387 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.317156 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:08.329086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.329054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:08.329236 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.329092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:08.329236 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:08.329214 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 16:57:08.329354 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:08.329268 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 16:57:08.329354 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:08.329290 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:09.329267613 +0000 UTC m=+41.092259468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : configmap "kube-rbac-proxy" not found Jul 08 16:57:08.329354 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:08.329314 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:09.329302709 +0000 UTC m=+41.092294558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : secret "insights-runtime-extractor-tls" not found Jul 08 16:57:08.418101 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.418062 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:08.429959 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.429928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:08.430103 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:08.430068 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 16:57:08.430169 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:08.430122 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls podName:3e40726d-a930-4435-9ba2-d7ede73e48a8 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:12.430107209 +0000 UTC m=+44.193099047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls") pod "dns-default-x7crb" (UID: "3e40726d-a930-4435-9ba2-d7ede73e48a8") : secret "dns-default-metrics-tls" not found Jul 08 16:57:08.657204 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.657080 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:08.766298 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.766263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:08.998126 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.998081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" event={"ID":"03e9c757-1324-4ebc-be00-bce7536d7986","Type":"ContainerStarted","Data":"fe47e08c0e16f88a95dded901c7ce6e229ec2c95558e1ede62b088897b16b3c4"} Jul 08 16:57:08.998541 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:08.998139 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" event={"ID":"03e9c757-1324-4ebc-be00-bce7536d7986","Type":"ContainerStarted","Data":"1aaeddd50f4589e9ffcaa7a1492c3c306f03b1c73e85e597af511e3c3c0a1823"} Jul 08 16:57:09.019672 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.019618 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vgwjr" podStartSLOduration=1.690487277 podStartE2EDuration="3.019602964s" totalImagesPullingTime="1.329115687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:09.01930667 +0000 UTC m=+40.782298531" watchObservedRunningTime="2026-07-08 16:57:09.019602964 +0000 UTC m=+40.782594824" Jul 08 16:57:09.153597 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.153563 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:09.246710 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.246680 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:09.335795 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.335763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:09.335941 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.335808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:09.335941 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:09.335841 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 16:57:09.335941 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:09.335916 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:11.335896859 +0000 UTC m=+43.098888706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : configmap "kube-rbac-proxy" not found Jul 08 16:57:09.336077 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:09.335949 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 16:57:09.336077 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:09.336004 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:11.335995385 +0000 UTC m=+43.098987228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : secret "insights-runtime-extractor-tls" not found Jul 08 16:57:09.511601 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.511464 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-wwkdm"] Jul 08 16:57:09.513381 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.513364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.518104 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.518076 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 16:57:09.518478 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.518456 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 16:57:09.519452 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.519435 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 16:57:09.519649 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.519634 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 16:57:09.519786 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.519769 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-grt8l\"" Jul 08 16:57:09.535058 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.535032 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-wwkdm"] Jul 08 16:57:09.537553 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.537530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-signing-cabundle\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.537662 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.537591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-signing-key\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.537662 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.537618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b94mq\" (UniqueName: \"kubernetes.io/projected/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-kube-api-access-b94mq\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.576664 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.576630 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:09.638641 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.638609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-signing-cabundle\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.638763 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.638657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-signing-key\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.638763 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.638685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b94mq\" (UniqueName: \"kubernetes.io/projected/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-kube-api-access-b94mq\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.639447 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.639425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-signing-cabundle\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.641461 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.641440 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-signing-key\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.648708 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.648681 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b94mq\" (UniqueName: \"kubernetes.io/projected/dedbd8e3-b84a-4eef-8678-af4bdea0d1f1-kube-api-access-b94mq\") pod \"service-ca-758f599c5-wwkdm\" (UID: \"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1\") " pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.679911 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.679873 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:09.821930 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.821844 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-wwkdm" Jul 08 16:57:09.945552 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:09.945237 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-wwkdm"] Jul 08 16:57:09.947796 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:09.947769 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedbd8e3_b84a_4eef_8678_af4bdea0d1f1.slice/crio-615ff82baeca4d8a6a6438af1f97744b8e7b849891d0725c4681adc67eeb0e02 WatchSource:0}: Error finding container 615ff82baeca4d8a6a6438af1f97744b8e7b849891d0725c4681adc67eeb0e02: Status 404 returned error can't find the container with id 615ff82baeca4d8a6a6438af1f97744b8e7b849891d0725c4681adc67eeb0e02 Jul 08 16:57:10.001193 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:10.001164 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-wwkdm" event={"ID":"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1","Type":"ContainerStarted","Data":"615ff82baeca4d8a6a6438af1f97744b8e7b849891d0725c4681adc67eeb0e02"} Jul 08 16:57:10.313563 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:10.313486 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bc62n_6007f44b-deb5-47d6-a9ba-53ea03fe32e6/dns-node-resolver/0.log" Jul 08 16:57:10.599436 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:10.599339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:11.133387 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:11.133360 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7tb8p_c752ba3a-30e7-4d5a-85f8-eea123b359aa/node-ca/0.log" Jul 08 16:57:11.223431 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:11.223393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:11.352018 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:11.351971 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:11.352018 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:11.352017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:11.352268 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:11.352221 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 16:57:11.352334 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:11.352283 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:15.352267936 +0000 UTC m=+47.115259775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : secret "insights-runtime-extractor-tls" not found Jul 08 16:57:11.352515 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:11.352497 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8fecbdd8-5abd-4b47-9e87-2957eda23b79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:12.114906 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:12.114875 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brkmd_dd0ce6a4-e00e-4f12-aba0-5524ccbac037/serve-healthcheck-canary/0.log" Jul 08 16:57:12.460435 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:12.460347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:12.460779 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:12.460503 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 16:57:12.460779 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:12.460566 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls podName:3e40726d-a930-4435-9ba2-d7ede73e48a8 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:20.460550527 +0000 UTC m=+52.223542365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls") pod "dns-default-x7crb" (UID: "3e40726d-a930-4435-9ba2-d7ede73e48a8") : secret "dns-default-metrics-tls" not found Jul 08 16:57:12.518530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:12.518504 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vgwjr_03e9c757-1324-4ebc-be00-bce7536d7986/migrator/0.log" Jul 08 16:57:12.713863 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:12.713793 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vgwjr_03e9c757-1324-4ebc-be00-bce7536d7986/graceful-termination/0.log" Jul 08 16:57:13.012218 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:13.012187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-wwkdm" event={"ID":"dedbd8e3-b84a-4eef-8678-af4bdea0d1f1","Type":"ContainerStarted","Data":"aebe7587596c8c085ca6898d3e0f73427bf99ff93770bc9303bb1c09b6d8d928"} Jul 08 16:57:13.033389 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:13.033331 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-wwkdm" podStartSLOduration=1.221821558 podStartE2EDuration="4.033315027s" totalImagesPullingTime="2.811493469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:13.033033712 +0000 UTC m=+44.796025572" watchObservedRunningTime="2026-07-08 16:57:13.033315027 +0000 UTC m=+44.796306887" Jul 08 16:57:14.333388 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:14.333355 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:14.505300 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:14.505265 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:14.604923 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:14.604826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:14.812115 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:14.812086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:14.911222 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:14.911075 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:15.173407 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:15.173315 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:15.271462 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:15.271423 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:15.383745 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:15.383707 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:15.384247 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:15.383892 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 16:57:15.384247 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:15.383976 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls podName:8fecbdd8-5abd-4b47-9e87-2957eda23b79 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:23.383952453 +0000 UTC m=+55.146944294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqtkk" (UID: "8fecbdd8-5abd-4b47-9e87-2957eda23b79") : secret "insights-runtime-extractor-tls" not found Jul 08 16:57:15.442050 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:15.441959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:15.542286 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:15.542257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:16.659804 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:16.659769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:16.847801 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:16.847761 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:17.067813 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:17.067774 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:17.257277 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:17.257246 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:17.442713 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:17.442621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:18.091110 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:18.091066 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:18.401523 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:18.401438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:18.502608 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:18.502574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:18.849208 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:18.849162 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:19.035468 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:19.035435 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:19.177184 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:19.177082 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:19.307493 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:19.307455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:19.468256 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:19.468125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:19.624574 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:19.624543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:19.743434 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:19.743322 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:20.437478 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:20.437444 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:20.522274 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:20.522241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:20.524524 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:20.524507 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e40726d-a930-4435-9ba2-d7ede73e48a8-metrics-tls\") pod \"dns-default-x7crb\" (UID: \"3e40726d-a930-4435-9ba2-d7ede73e48a8\") " pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:20.538070 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:20.538027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:20.579569 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:20.579494 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:20.728129 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:20.728051 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x7crb"] Jul 08 16:57:20.731103 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:20.731080 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e40726d_a930_4435_9ba2_d7ede73e48a8.slice/crio-061faafadff28a9fa69be85154ec42780952a05def9e42e5ad39f7663f85ac50 WatchSource:0}: Error finding container 061faafadff28a9fa69be85154ec42780952a05def9e42e5ad39f7663f85ac50: Status 404 returned error can't find the container with id 061faafadff28a9fa69be85154ec42780952a05def9e42e5ad39f7663f85ac50 Jul 08 16:57:21.032860 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:21.032830 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x7crb" event={"ID":"3e40726d-a930-4435-9ba2-d7ede73e48a8","Type":"ContainerStarted","Data":"061faafadff28a9fa69be85154ec42780952a05def9e42e5ad39f7663f85ac50"} Jul 08 16:57:22.486289 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:22.486247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:22.605020 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:22.604984 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:22.832682 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:22.832646 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:23.018439 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.018393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:23.237325 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.237288 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:23.424728 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.424629 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:23.444661 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.444627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:23.446902 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.446879 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8fecbdd8-5abd-4b47-9e87-2957eda23b79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqtkk\" (UID: \"8fecbdd8-5abd-4b47-9e87-2957eda23b79\") " pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:23.459928 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.459894 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqtkk" Jul 08 16:57:23.609476 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.609440 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqtkk"] Jul 08 16:57:23.612997 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:23.612976 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fecbdd8_5abd_4b47_9e87_2957eda23b79.slice/crio-27ee213edbfc58675d3bd747edc122eeab736a11ab63e9375a46d26b0cba6838 WatchSource:0}: Error finding container 27ee213edbfc58675d3bd747edc122eeab736a11ab63e9375a46d26b0cba6838: Status 404 returned error can't find the container with id 27ee213edbfc58675d3bd747edc122eeab736a11ab63e9375a46d26b0cba6838 Jul 08 16:57:23.640712 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:23.640673 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:24.043859 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.043819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x7crb" event={"ID":"3e40726d-a930-4435-9ba2-d7ede73e48a8","Type":"ContainerStarted","Data":"73722fa705a464000128e8f1b1d86c3e91978a87a372129d0989b38b3db4bdaf"} Jul 08 16:57:24.044053 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.043868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x7crb" event={"ID":"3e40726d-a930-4435-9ba2-d7ede73e48a8","Type":"ContainerStarted","Data":"ea265e649088da6f2bc666c6166036e4e8d58941fd53e5a65f3061898693a4ca"} Jul 08 16:57:24.044053 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.043953 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:24.044986 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.044965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqtkk" event={"ID":"8fecbdd8-5abd-4b47-9e87-2957eda23b79","Type":"ContainerStarted","Data":"27ee213edbfc58675d3bd747edc122eeab736a11ab63e9375a46d26b0cba6838"} Jul 08 16:57:24.065383 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.065331 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x7crb" podStartSLOduration=17.880643808 podStartE2EDuration="20.065315436s" totalImagesPullingTime="2.184671628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:24.065224281 +0000 UTC m=+55.828216138" watchObservedRunningTime="2026-07-08 16:57:24.065315436 +0000 UTC m=+55.828307295" Jul 08 16:57:24.219458 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.219427 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:24.618050 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.617985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:24.815285 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:24.815255 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:25.051371 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:25.051337 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:27.159666 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:27.159624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:27.297515 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:27.297456 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:29.779829 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:29.779792 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:29.880256 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:29.880219 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:29.999361 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:29.999331 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:30.068017 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.067934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqtkk" event={"ID":"8fecbdd8-5abd-4b47-9e87-2957eda23b79","Type":"ContainerStarted","Data":"0dd65afafa97c5e4adfa4e86de431d97798db195aacb204ad3e95509272fa729"} Jul 08 16:57:30.068017 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.068011 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqtkk" event={"ID":"8fecbdd8-5abd-4b47-9e87-2957eda23b79","Type":"ContainerStarted","Data":"f94b3811f41e892f610728ac4033b8840723c3cbdf2240181bf87413819e0a09"} Jul 08 16:57:30.195742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.195698 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:30.370486 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.370412 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:30.465111 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.465070 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:30.577810 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.577772 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:30.975224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:30.975195 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-px95b" Jul 08 16:57:31.434915 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:31.434878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:31.556760 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:31.556720 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:31.667599 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:31.667553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:32.063871 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.063837 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm"] Jul 08 16:57:32.070869 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.070842 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.075197 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.074970 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 16:57:32.075197 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.075049 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 16:57:32.075197 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.074977 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 16:57:32.075691 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.075672 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 16:57:32.075799 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.075750 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-t7vb7\"" Jul 08 16:57:32.106166 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.106110 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm"] Jul 08 16:57:32.118268 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.118233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc80fc85-1129-42c5-82e6-501bd2ef0c7e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm\" (UID: \"cc80fc85-1129-42c5-82e6-501bd2ef0c7e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.118440 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.118286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k78nt\" (UniqueName: \"kubernetes.io/projected/cc80fc85-1129-42c5-82e6-501bd2ef0c7e-kube-api-access-k78nt\") pod \"managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm\" (UID: \"cc80fc85-1129-42c5-82e6-501bd2ef0c7e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.157693 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.157644 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-zjlt6"] Jul 08 16:57:32.168163 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.168117 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:32.172448 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.172313 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gg2x6\"" Jul 08 16:57:32.172448 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.172323 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 16:57:32.172448 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.172375 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 16:57:32.185356 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.185325 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-zjlt6"] Jul 08 16:57:32.219615 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.219578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc80fc85-1129-42c5-82e6-501bd2ef0c7e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm\" (UID: \"cc80fc85-1129-42c5-82e6-501bd2ef0c7e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.219794 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.219627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k78nt\" (UniqueName: \"kubernetes.io/projected/cc80fc85-1129-42c5-82e6-501bd2ef0c7e-kube-api-access-k78nt\") pod \"managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm\" (UID: \"cc80fc85-1129-42c5-82e6-501bd2ef0c7e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.219794 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.219724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h79v2\" (UniqueName: \"kubernetes.io/projected/3090f41d-918d-42ac-b7ae-837a2cb25685-kube-api-access-h79v2\") pod \"downloads-685bfb5f64-zjlt6\" (UID: \"3090f41d-918d-42ac-b7ae-837a2cb25685\") " pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:32.222393 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.222365 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc80fc85-1129-42c5-82e6-501bd2ef0c7e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm\" (UID: \"cc80fc85-1129-42c5-82e6-501bd2ef0c7e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.246228 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.246198 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k78nt\" (UniqueName: \"kubernetes.io/projected/cc80fc85-1129-42c5-82e6-501bd2ef0c7e-kube-api-access-k78nt\") pod \"managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm\" (UID: \"cc80fc85-1129-42c5-82e6-501bd2ef0c7e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.284027 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.283992 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:32.291561 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.291534 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7985fdcb94-hbsms"] Jul 08 16:57:32.295909 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.295889 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.300439 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.300411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 16:57:32.300569 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.300460 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jtpwm\"" Jul 08 16:57:32.300628 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.300612 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 16:57:32.300916 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.300902 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 16:57:32.307010 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.306987 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 16:57:32.316195 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.316102 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7985fdcb94-hbsms"] Jul 08 16:57:32.320504 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.320477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h79v2\" (UniqueName: \"kubernetes.io/projected/3090f41d-918d-42ac-b7ae-837a2cb25685-kube-api-access-h79v2\") pod \"downloads-685bfb5f64-zjlt6\" (UID: \"3090f41d-918d-42ac-b7ae-837a2cb25685\") " pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:32.335639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.335613 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h79v2\" (UniqueName: \"kubernetes.io/projected/3090f41d-918d-42ac-b7ae-837a2cb25685-kube-api-access-h79v2\") pod \"downloads-685bfb5f64-zjlt6\" (UID: \"3090f41d-918d-42ac-b7ae-837a2cb25685\") " pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:32.391199 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.391163 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" Jul 08 16:57:32.398325 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.398295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:32.421818 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.421788 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-registry-tls\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.421988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.421843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0588eff6-ffcc-4062-9bad-cc308888dc9f-installation-pull-secrets\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.421988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.421868 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55nj4\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-kube-api-access-55nj4\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.421988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.421887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0588eff6-ffcc-4062-9bad-cc308888dc9f-image-registry-private-configuration\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.421988 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.421904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0588eff6-ffcc-4062-9bad-cc308888dc9f-ca-trust-extracted\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.422132 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.422025 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0588eff6-ffcc-4062-9bad-cc308888dc9f-registry-certificates\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.422132 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.422064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-bound-sa-token\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.422216 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.422157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0588eff6-ffcc-4062-9bad-cc308888dc9f-trusted-ca\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.479425 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.479395 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:32.498165 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.498072 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:32.523003 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.522973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0588eff6-ffcc-4062-9bad-cc308888dc9f-registry-certificates\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523003 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-bound-sa-token\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523042 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0588eff6-ffcc-4062-9bad-cc308888dc9f-trusted-ca\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523074 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-registry-tls\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523097 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0588eff6-ffcc-4062-9bad-cc308888dc9f-installation-pull-secrets\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55nj4\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-kube-api-access-55nj4\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0588eff6-ffcc-4062-9bad-cc308888dc9f-image-registry-private-configuration\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0588eff6-ffcc-4062-9bad-cc308888dc9f-ca-trust-extracted\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.523550 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523515 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0588eff6-ffcc-4062-9bad-cc308888dc9f-ca-trust-extracted\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.524005 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.523952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0588eff6-ffcc-4062-9bad-cc308888dc9f-registry-certificates\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.524680 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.524644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0588eff6-ffcc-4062-9bad-cc308888dc9f-trusted-ca\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.525775 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.525756 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-registry-tls\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.526355 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.526337 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0588eff6-ffcc-4062-9bad-cc308888dc9f-installation-pull-secrets\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.526656 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.526634 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0588eff6-ffcc-4062-9bad-cc308888dc9f-image-registry-private-configuration\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.536048 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.536017 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-bound-sa-token\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.536336 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.536311 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55nj4\" (UniqueName: \"kubernetes.io/projected/0588eff6-ffcc-4062-9bad-cc308888dc9f-kube-api-access-55nj4\") pod \"image-registry-7985fdcb94-hbsms\" (UID: \"0588eff6-ffcc-4062-9bad-cc308888dc9f\") " pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.601293 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.601255 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm"] Jul 08 16:57:32.602784 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.602748 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:32.604644 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:32.604615 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc80fc85_1129_42c5_82e6_501bd2ef0c7e.slice/crio-3cf63f5fb8cb8db6bcd02f2df9ddf10060a88099f82f5a379c93e9f016da61f2 WatchSource:0}: Error finding container 3cf63f5fb8cb8db6bcd02f2df9ddf10060a88099f82f5a379c93e9f016da61f2: Status 404 returned error can't find the container with id 3cf63f5fb8cb8db6bcd02f2df9ddf10060a88099f82f5a379c93e9f016da61f2 Jul 08 16:57:32.608046 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.608026 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:32.640990 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.640963 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-zjlt6"] Jul 08 16:57:32.643759 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:32.643730 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3090f41d_918d_42ac_b7ae_837a2cb25685.slice/crio-c7a47433e46b50ce858d96efc4627aa9e40ca64f122d449eb063495eb1b73e4c WatchSource:0}: Error finding container c7a47433e46b50ce858d96efc4627aa9e40ca64f122d449eb063495eb1b73e4c: Status 404 returned error can't find the container with id c7a47433e46b50ce858d96efc4627aa9e40ca64f122d449eb063495eb1b73e4c Jul 08 16:57:32.744354 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.744179 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7985fdcb94-hbsms"] Jul 08 16:57:32.746761 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:32.746726 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0588eff6_ffcc_4062_9bad_cc308888dc9f.slice/crio-31cff73d5a7f38f8acedd963eeed71d22986972e8a4103f466870d45e2210710 WatchSource:0}: Error finding container 31cff73d5a7f38f8acedd963eeed71d22986972e8a4103f466870d45e2210710: Status 404 returned error can't find the container with id 31cff73d5a7f38f8acedd963eeed71d22986972e8a4103f466870d45e2210710 Jul 08 16:57:32.880874 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.880794 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:32.949919 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:32.949880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:33.078992 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.078954 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-zjlt6" event={"ID":"3090f41d-918d-42ac-b7ae-837a2cb25685","Type":"ContainerStarted","Data":"c7a47433e46b50ce858d96efc4627aa9e40ca64f122d449eb063495eb1b73e4c"} Jul 08 16:57:33.081049 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.081018 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqtkk" event={"ID":"8fecbdd8-5abd-4b47-9e87-2957eda23b79","Type":"ContainerStarted","Data":"f1dd4f9e7925a9661935e9122df208933362a4a51a449a6c22749996b28bc73c"} Jul 08 16:57:33.082502 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.082481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" event={"ID":"0588eff6-ffcc-4062-9bad-cc308888dc9f","Type":"ContainerStarted","Data":"22b4bfc9fc2fbfef18e7d0d132fce6868d74f08b4c4401313b922225017dfb1b"} Jul 08 16:57:33.082609 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.082520 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" event={"ID":"0588eff6-ffcc-4062-9bad-cc308888dc9f","Type":"ContainerStarted","Data":"31cff73d5a7f38f8acedd963eeed71d22986972e8a4103f466870d45e2210710"} Jul 08 16:57:33.082609 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.082544 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:33.083651 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.083629 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" event={"ID":"cc80fc85-1129-42c5-82e6-501bd2ef0c7e","Type":"ContainerStarted","Data":"3cf63f5fb8cb8db6bcd02f2df9ddf10060a88099f82f5a379c93e9f016da61f2"} Jul 08 16:57:33.101409 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.101347 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zqtkk" podStartSLOduration=17.361030071 podStartE2EDuration="26.101327617s" totalImagesPullingTime="8.740297546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:57:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:33.101281084 +0000 UTC m=+64.864272944" watchObservedRunningTime="2026-07-08 16:57:33.101327617 +0000 UTC m=+64.864319478" Jul 08 16:57:33.123755 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:33.123707 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" podStartSLOduration=1.123687899 podStartE2EDuration="1.123687899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:33.122556159 +0000 UTC m=+64.885548019" watchObservedRunningTime="2026-07-08 16:57:33.123687899 +0000 UTC m=+64.886679758" Jul 08 16:57:34.050862 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.050832 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x7crb" Jul 08 16:57:34.543023 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.542991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:34.543473 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.543056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:34.550670 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.550645 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 16:57:34.550798 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.550747 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 16:57:34.556099 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.556068 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687-metrics-certs\") pod \"network-metrics-daemon-nkwpj\" (UID: \"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687\") " pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:34.556475 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.556455 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec13312c-4802-428d-bb87-0a70978eb388-original-pull-secret\") pod \"global-pull-secret-syncer-lgbxp\" (UID: \"ec13312c-4802-428d-bb87-0a70978eb388\") " pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:34.582395 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.582360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:34.609881 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.609852 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgbxp" Jul 08 16:57:34.618485 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.618463 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pgff9\"" Jul 08 16:57:34.625976 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.625955 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nkwpj" Jul 08 16:57:34.644379 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.644353 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:34.647065 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.647044 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 16:57:34.657334 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.657310 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 16:57:34.668331 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.668303 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqj5d\" (UniqueName: \"kubernetes.io/projected/12a12eb7-e5a5-42f0-9a59-0c70f6e87268-kube-api-access-cqj5d\") pod \"network-check-target-pb29q\" (UID: \"12a12eb7-e5a5-42f0-9a59-0c70f6e87268\") " pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:34.718015 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.717945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:34.788696 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.788664 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w"] Jul 08 16:57:34.790386 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.790363 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:34.794526 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.794468 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-868sd\"" Jul 08 16:57:34.794800 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.794778 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 16:57:34.806547 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.806525 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w"] Jul 08 16:57:34.822944 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.822835 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:34.845916 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.845881 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c17e049-6ed0-4703-b8ec-b68606678591-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7vx6w\" (UID: \"6c17e049-6ed0-4703-b8ec-b68606678591\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:34.905571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.905538 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-594z6\"" Jul 08 16:57:34.913617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.913586 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:34.947001 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:34.946965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c17e049-6ed0-4703-b8ec-b68606678591-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7vx6w\" (UID: \"6c17e049-6ed0-4703-b8ec-b68606678591\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:34.947186 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:34.947125 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 08 16:57:34.947257 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:57:34.947206 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c17e049-6ed0-4703-b8ec-b68606678591-tls-certificates podName:6c17e049-6ed0-4703-b8ec-b68606678591 nodeName:}" failed. No retries permitted until 2026-07-08 16:57:35.447183033 +0000 UTC m=+67.210174874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6c17e049-6ed0-4703-b8ec-b68606678591-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-7vx6w" (UID: "6c17e049-6ed0-4703-b8ec-b68606678591") : secret "prometheus-operator-admission-webhook-tls" not found Jul 08 16:57:35.183664 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.183593 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:35.386088 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.385977 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lgbxp"] Jul 08 16:57:35.388829 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:35.388790 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec13312c_4802_428d_bb87_0a70978eb388.slice/crio-4427a0172f87efd3a980dc4d9ee89f3c0f3733cde5e5c789f25f5cb78858b717 WatchSource:0}: Error finding container 4427a0172f87efd3a980dc4d9ee89f3c0f3733cde5e5c789f25f5cb78858b717: Status 404 returned error can't find the container with id 4427a0172f87efd3a980dc4d9ee89f3c0f3733cde5e5c789f25f5cb78858b717 Jul 08 16:57:35.401123 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.401100 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nkwpj"] Jul 08 16:57:35.403822 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:35.403767 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747c5ae0_7f2c_4cfd_81e1_c8dbecf1f687.slice/crio-e39e2ed0e105b2366dd5c396f5f9263ad43f7ff6f7bd717b5c3dfd451405dc8f WatchSource:0}: Error finding container e39e2ed0e105b2366dd5c396f5f9263ad43f7ff6f7bd717b5c3dfd451405dc8f: Status 404 returned error can't find the container with id e39e2ed0e105b2366dd5c396f5f9263ad43f7ff6f7bd717b5c3dfd451405dc8f Jul 08 16:57:35.426913 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.426880 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pb29q"] Jul 08 16:57:35.429964 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:35.429938 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a12eb7_e5a5_42f0_9a59_0c70f6e87268.slice/crio-8696cabfcb36ee939b07bdb6bf1e4f3b6a68ca9129ebb29e545ea574e502ba24 WatchSource:0}: Error finding container 8696cabfcb36ee939b07bdb6bf1e4f3b6a68ca9129ebb29e545ea574e502ba24: Status 404 returned error can't find the container with id 8696cabfcb36ee939b07bdb6bf1e4f3b6a68ca9129ebb29e545ea574e502ba24 Jul 08 16:57:35.451390 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.451357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c17e049-6ed0-4703-b8ec-b68606678591-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7vx6w\" (UID: \"6c17e049-6ed0-4703-b8ec-b68606678591\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:35.454109 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.454086 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6c17e049-6ed0-4703-b8ec-b68606678591-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7vx6w\" (UID: \"6c17e049-6ed0-4703-b8ec-b68606678591\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:35.700787 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.700708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:35.848327 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:35.848298 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w"] Jul 08 16:57:35.851580 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:35.851550 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c17e049_6ed0_4703_b8ec_b68606678591.slice/crio-88fe746fc98ccb2db3007fa264f3aae7898b47fb65ccc3a16823797efa2e8bfd WatchSource:0}: Error finding container 88fe746fc98ccb2db3007fa264f3aae7898b47fb65ccc3a16823797efa2e8bfd: Status 404 returned error can't find the container with id 88fe746fc98ccb2db3007fa264f3aae7898b47fb65ccc3a16823797efa2e8bfd Jul 08 16:57:36.097814 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:36.097774 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pb29q" event={"ID":"12a12eb7-e5a5-42f0-9a59-0c70f6e87268","Type":"ContainerStarted","Data":"8696cabfcb36ee939b07bdb6bf1e4f3b6a68ca9129ebb29e545ea574e502ba24"} Jul 08 16:57:36.099872 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:36.099513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" event={"ID":"cc80fc85-1129-42c5-82e6-501bd2ef0c7e","Type":"ContainerStarted","Data":"b1cf9e13be8bf0c039850084371456de7846bd1056953fc65aca9dae3a594ca2"} Jul 08 16:57:36.101205 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:36.101181 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nkwpj" event={"ID":"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687","Type":"ContainerStarted","Data":"e39e2ed0e105b2366dd5c396f5f9263ad43f7ff6f7bd717b5c3dfd451405dc8f"} Jul 08 16:57:36.102663 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:36.102638 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" event={"ID":"6c17e049-6ed0-4703-b8ec-b68606678591","Type":"ContainerStarted","Data":"88fe746fc98ccb2db3007fa264f3aae7898b47fb65ccc3a16823797efa2e8bfd"} Jul 08 16:57:36.104176 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:36.104137 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lgbxp" event={"ID":"ec13312c-4802-428d-bb87-0a70978eb388","Type":"ContainerStarted","Data":"4427a0172f87efd3a980dc4d9ee89f3c0f3733cde5e5c789f25f5cb78858b717"} Jul 08 16:57:36.120305 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:36.120249 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c6f9b6c6b-m6hrm" podStartSLOduration=1.504586288 podStartE2EDuration="4.120230236s" totalImagesPullingTime="2.615643948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:36.119743846 +0000 UTC m=+67.882735707" watchObservedRunningTime="2026-07-08 16:57:36.120230236 +0000 UTC m=+67.883222097" Jul 08 16:57:37.671064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:37.670825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:37.772001 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:37.771964 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:38.117336 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.117302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" event={"ID":"6c17e049-6ed0-4703-b8ec-b68606678591","Type":"ContainerStarted","Data":"d7dc0f29ee69ebd9c0669b80ff3a0e744d4d2fff351849a5231dc62ebab9e30b"} Jul 08 16:57:38.118281 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.118246 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:38.125869 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.125839 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" Jul 08 16:57:38.139561 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.138944 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7vx6w" podStartSLOduration=2.692766281 podStartE2EDuration="4.138921005s" totalImagesPullingTime="1.446154724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:38.13675147 +0000 UTC m=+69.899743331" watchObservedRunningTime="2026-07-08 16:57:38.138921005 +0000 UTC m=+69.901912866" Jul 08 16:57:38.221396 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.221309 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:38.375343 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.375251 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:38.820619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.820510 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:38.951463 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:38.951359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:39.124545 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.124511 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nkwpj" event={"ID":"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687","Type":"ContainerStarted","Data":"2f2ad09e29451ac6b34aa9b635bad9f864b2818827e1df093ce6baedaf737c54"} Jul 08 16:57:39.210881 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.210793 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:39.448789 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.448753 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-697fd5d8cc-5r9lr"] Jul 08 16:57:39.483742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.482296 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697fd5d8cc-5r9lr"] Jul 08 16:57:39.483742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.482416 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.488910 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.488883 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ltj2g\"" Jul 08 16:57:39.491162 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.490970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 16:57:39.491162 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.491069 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 16:57:39.492709 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.492688 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 16:57:39.492812 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.492744 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 16:57:39.492983 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.492962 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 16:57:39.589774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.589470 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-oauth-config\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.589774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.589521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-service-ca\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.589774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.589557 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-oauth-serving-cert\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.589774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.589587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-console-config\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.589774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.589619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-serving-cert\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.589774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.589668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktzqm\" (UniqueName: \"kubernetes.io/projected/165b7775-49ae-4aa6-85de-8a4668a958a5-kube-api-access-ktzqm\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.690416 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.690373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktzqm\" (UniqueName: \"kubernetes.io/projected/165b7775-49ae-4aa6-85de-8a4668a958a5-kube-api-access-ktzqm\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.690587 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.690431 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-oauth-config\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.690587 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.690473 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-service-ca\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.690587 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.690499 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-oauth-serving-cert\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.690587 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.690522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-console-config\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.690587 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.690547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-serving-cert\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.691382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.691357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-console-config\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.691489 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.691357 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-oauth-serving-cert\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.691563 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.691541 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-service-ca\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.693536 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.693512 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-serving-cert\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.693630 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.693538 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-oauth-config\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.700227 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.700194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktzqm\" (UniqueName: \"kubernetes.io/projected/165b7775-49ae-4aa6-85de-8a4668a958a5-kube-api-access-ktzqm\") pod \"console-697fd5d8cc-5r9lr\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.795939 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.795899 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:57:39.845126 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.845086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:39.958955 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:39.958916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:40.695179 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:40.695125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:40.824235 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:40.824172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:41.848025 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:41.847963 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:41.988333 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:41.988301 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697fd5d8cc-5r9lr"] Jul 08 16:57:41.992024 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:41.991997 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165b7775_49ae_4aa6_85de_8a4668a958a5.slice/crio-cf2bf9c9a5b5a9557f42e0945f228f4bb4d582048d76b3befa390398b1565484 WatchSource:0}: Error finding container cf2bf9c9a5b5a9557f42e0945f228f4bb4d582048d76b3befa390398b1565484: Status 404 returned error can't find the container with id cf2bf9c9a5b5a9557f42e0945f228f4bb4d582048d76b3befa390398b1565484 Jul 08 16:57:42.139186 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.139134 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lgbxp" event={"ID":"ec13312c-4802-428d-bb87-0a70978eb388","Type":"ContainerStarted","Data":"eb08241d947df7f61dc2c7948f41edd6d2dfc0fb9fb77979aa945d4aa6fda8bc"} Jul 08 16:57:42.140818 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.140791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pb29q" event={"ID":"12a12eb7-e5a5-42f0-9a59-0c70f6e87268","Type":"ContainerStarted","Data":"53a950b4408e6e14b01f6b2d3f7027d0723a6be2baca73f3e2acd73a8b3867cd"} Jul 08 16:57:42.140949 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.140841 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:57:42.142837 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.142705 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nkwpj" event={"ID":"747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687","Type":"ContainerStarted","Data":"d746a5af8bb8b257068b0e2cccde465ecdf8c5ae83b97c1d26a8c4be3ebcf9ec"} Jul 08 16:57:42.144136 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.144113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd5d8cc-5r9lr" event={"ID":"165b7775-49ae-4aa6-85de-8a4668a958a5","Type":"ContainerStarted","Data":"cf2bf9c9a5b5a9557f42e0945f228f4bb4d582048d76b3befa390398b1565484"} Jul 08 16:57:42.170081 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.169940 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lgbxp" podStartSLOduration=66.753221906 podStartE2EDuration="1m13.169921432s" totalImagesPullingTime="6.416699526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:42.168839222 +0000 UTC m=+73.931831083" watchObservedRunningTime="2026-07-08 16:57:42.169921432 +0000 UTC m=+73.932913296" Jul 08 16:57:42.203604 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.203542 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nkwpj" podStartSLOduration=70.019667066 podStartE2EDuration="1m13.203512554s" totalImagesPullingTime="3.183845488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:42.203324815 +0000 UTC m=+73.966316676" watchObservedRunningTime="2026-07-08 16:57:42.203512554 +0000 UTC m=+73.966504412" Jul 08 16:57:42.617788 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.617734 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:42.836260 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:42.836211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:43.589201 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:43.589169 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:43.848811 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:43.848729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:44.467115 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:44.467080 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:45.074542 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.074506 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:45.309179 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.309121 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:45.732802 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.732743 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pb29q" podStartSLOduration=70.37561863 podStartE2EDuration="1m16.732721011s" totalImagesPullingTime="6.357102381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:42.289659126 +0000 UTC m=+74.052650986" watchObservedRunningTime="2026-07-08 16:57:45.732721011 +0000 UTC m=+77.495712951" Jul 08 16:57:45.734127 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.734107 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-smmh6"] Jul 08 16:57:45.739803 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.739778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.760693 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.760646 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:45.783779 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.783749 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 16:57:45.791318 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.791299 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 16:57:45.793282 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.793265 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 16:57:45.793560 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.793536 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 16:57:45.794357 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.794339 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 16:57:45.809164 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.809117 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 16:57:45.809505 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.809489 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2mgbd\"" Jul 08 16:57:45.847330 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-textfile\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847330 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847303 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-tls\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847330 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847330 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-wtmp\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847402 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dbf3d66-3020-4891-af4b-06f1b4b072d8-metrics-client-ca\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847465 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847486 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f99p\" (UniqueName: \"kubernetes.io/projected/3dbf3d66-3020-4891-af4b-06f1b4b072d8-kube-api-access-8f99p\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847550 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-sys\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-root\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.847619 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.847582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948198 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948198 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948194 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f99p\" (UniqueName: \"kubernetes.io/projected/3dbf3d66-3020-4891-af4b-06f1b4b072d8-kube-api-access-8f99p\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-sys\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-root\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-textfile\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-tls\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948361 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-wtmp\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948420 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948390 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dbf3d66-3020-4891-af4b-06f1b4b072d8-metrics-client-ca\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948700 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948673 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-root\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948773 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948750 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-textfile\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.948809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.948337 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-sys\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.949030 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.949005 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dbf3d66-3020-4891-af4b-06f1b4b072d8-metrics-client-ca\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.949177 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.949140 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-wtmp\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.949253 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.949224 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.951050 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.951027 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:45.951256 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:45.951235 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dbf3d66-3020-4891-af4b-06f1b4b072d8-node-exporter-tls\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:46.002314 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.002245 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f99p\" (UniqueName: \"kubernetes.io/projected/3dbf3d66-3020-4891-af4b-06f1b4b072d8-kube-api-access-8f99p\") pod \"node-exporter-smmh6\" (UID: \"3dbf3d66-3020-4891-af4b-06f1b4b072d8\") " pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:46.050491 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.050458 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-smmh6" Jul 08 16:57:46.114193 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.114116 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:46.565757 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.565625 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:46.861951 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.861794 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:57:46.864855 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.864832 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.872882 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.872807 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 16:57:46.872882 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.872811 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 16:57:46.872882 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.872863 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 16:57:46.873748 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.873689 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 16:57:46.873748 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.873712 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dchxq\"" Jul 08 16:57:46.873907 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.873750 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 16:57:46.873907 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.873753 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 16:57:46.874028 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.874012 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 16:57:46.877553 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.877536 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 16:57:46.887491 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.887463 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:46.888697 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.888676 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 16:57:46.929320 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.929285 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:57:46.958111 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-volume\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958111 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958115 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-web-config\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958340 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958142 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958340 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958250 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958340 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958461 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958347 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958461 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958461 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958461 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958461 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958662 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958516 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqgwv\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-kube-api-access-dqgwv\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958662 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-out\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:46.958662 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:46.958598 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059652 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059610 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059662 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqgwv\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-kube-api-access-dqgwv\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.059841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059831 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-out\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059891 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-volume\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-web-config\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.059983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.060774 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.060734 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.061057 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.060956 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.061739 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.061708 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.063064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.063044 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.063892 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.063847 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-web-config\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.064006 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.063983 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.064326 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.064277 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-out\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.064425 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.064331 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.064899 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.064873 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.065479 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.065461 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-volume\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.066015 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.065996 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.066441 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.066424 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.088316 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.088290 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqgwv\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-kube-api-access-dqgwv\") pod \"alertmanager-main-0\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.176137 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.176046 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:57:47.346380 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:47.346346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:49.231233 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:49.231179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:49.565039 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:49.564981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:49.841001 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:49.840902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:50.066184 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:50.066048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:50.305233 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:50.305196 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:51.007045 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.007017 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf"] Jul 08 16:57:51.118683 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.118651 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf"] Jul 08 16:57:51.118936 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.118758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.126648 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.126627 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 16:57:51.126837 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.126816 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 16:57:51.127077 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.127054 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 16:57:51.127748 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.127731 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 16:57:51.127853 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.127787 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-p9d48\"" Jul 08 16:57:51.127853 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.127807 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 16:57:51.130870 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.130850 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 16:57:51.298138 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298107 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf9gx\" (UniqueName: \"kubernetes.io/projected/5cd01afb-1f34-4bc0-b447-715f5df04076-kube-api-access-xf9gx\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298339 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-secret-telemeter-client\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298339 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-federate-client-tls\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298339 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298339 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298296 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-metrics-client-ca\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-serving-certs-ca-bundle\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.298517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.298418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-telemeter-client-tls\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.399721 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.399682 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-metrics-client-ca\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.399740 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-serving-certs-ca-bundle\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.399784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-telemeter-client-tls\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.399807 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf9gx\" (UniqueName: \"kubernetes.io/projected/5cd01afb-1f34-4bc0-b447-715f5df04076-kube-api-access-xf9gx\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.399854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-secret-telemeter-client\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.399910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-federate-client-tls\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.400073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400283 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.400112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400713 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.400689 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-metrics-client-ca\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400786 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.400692 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-serving-certs-ca-bundle\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.400994 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.400974 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cd01afb-1f34-4bc0-b447-715f5df04076-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.403511 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.403489 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-telemeter-client-tls\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.403597 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.403564 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-federate-client-tls\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.403723 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.403708 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-secret-telemeter-client\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.403808 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.403786 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd01afb-1f34-4bc0-b447-715f5df04076-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.418475 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.418444 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf9gx\" (UniqueName: \"kubernetes.io/projected/5cd01afb-1f34-4bc0-b447-715f5df04076-kube-api-access-xf9gx\") pod \"telemeter-client-78d56bcd6c-qlhrf\" (UID: \"5cd01afb-1f34-4bc0-b447-715f5df04076\") " pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:51.429224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:51.429198 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" Jul 08 16:57:52.040622 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:52.040574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:52.413387 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:52.413299 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:52.574985 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:52.574952 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:53.095643 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:53.095588 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbf3d66_3020_4891_af4b_06f1b4b072d8.slice/crio-1b1ef4bd4241497def1377ffcc5b88b3db73413d1fd7041c23a1821baea1bad4 WatchSource:0}: Error finding container 1b1ef4bd4241497def1377ffcc5b88b3db73413d1fd7041c23a1821baea1bad4: Status 404 returned error can't find the container with id 1b1ef4bd4241497def1377ffcc5b88b3db73413d1fd7041c23a1821baea1bad4 Jul 08 16:57:53.188951 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.188923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smmh6" event={"ID":"3dbf3d66-3020-4891-af4b-06f1b4b072d8","Type":"ContainerStarted","Data":"1b1ef4bd4241497def1377ffcc5b88b3db73413d1fd7041c23a1821baea1bad4"} Jul 08 16:57:53.244696 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.244669 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:57:53.262412 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.262381 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:53.268415 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.268386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.280291 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.280272 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 16:57:53.284789 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.284765 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 16:57:53.284926 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.284907 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 16:57:53.285010 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.284992 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 16:57:53.285073 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.285053 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 16:57:53.285135 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.285082 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 16:57:53.291524 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.291475 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 16:57:53.291634 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.291536 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xgrmw\"" Jul 08 16:57:53.291742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.291656 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 16:57:53.291742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.291699 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 16:57:53.291742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.291757 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 16:57:53.292084 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.292049 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 16:57:53.292209 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.292189 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 16:57:53.292340 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.292322 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 16:57:53.292427 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.292383 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8jvfnuh9jo1tb\"" Jul 08 16:57:53.293835 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.293814 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 16:57:53.302328 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.302307 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 16:57:53.313285 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.313168 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf"] Jul 08 16:57:53.315622 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:53.315599 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba51c63_7b7f_40de_9127_21f27e86c7e9.slice/crio-914a0af8389e9d5da5df9a1d7468637e2cde5fddcfbc562d1bb23730938edd0d WatchSource:0}: Error finding container 914a0af8389e9d5da5df9a1d7468637e2cde5fddcfbc562d1bb23730938edd0d: Status 404 returned error can't find the container with id 914a0af8389e9d5da5df9a1d7468637e2cde5fddcfbc562d1bb23730938edd0d Jul 08 16:57:53.316342 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:57:53.316319 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd01afb_1f34_4bc0_b447_715f5df04076.slice/crio-490022063d17c9de54e2ecba44da80301c2531f9ae669991e7358546e8a251b4 WatchSource:0}: Error finding container 490022063d17c9de54e2ecba44da80301c2531f9ae669991e7358546e8a251b4: Status 404 returned error can't find the container with id 490022063d17c9de54e2ecba44da80301c2531f9ae669991e7358546e8a251b4 Jul 08 16:57:53.320558 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.320389 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:57:53.340779 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.340750 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:57:53.417724 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.417724 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417745 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417827 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config-out\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9p6\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-kube-api-access-xp9p6\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417911 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.417964 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418020 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418051 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418186 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418220 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-web-config\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.418633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.418281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519114 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519314 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519122 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519314 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519314 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519314 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519354 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519376 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-web-config\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519427 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519530 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519510 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519539 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519604 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config-out\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9p6\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-kube-api-access-xp9p6\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519727 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519797 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.519929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.519903 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.520612 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.520577 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.520978 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.520838 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.521382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.521358 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.522424 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.521897 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.523471 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.522767 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-web-config\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.523471 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.522882 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.524830 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.524804 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.525273 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.525248 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.525378 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.525354 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.525766 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.525741 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.525872 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.525848 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.525971 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.525951 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.526480 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.526308 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.526480 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.526337 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config-out\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.526480 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.526310 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.527187 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.527164 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.527831 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.527811 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.684734 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.684619 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9p6\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-kube-api-access-xp9p6\") pod \"prometheus-k8s-0\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:53.878638 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:53.878329 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:57:54.094788 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.094757 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7985fdcb94-hbsms" Jul 08 16:57:54.130473 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.130428 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:57:54.198505 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.198356 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-zjlt6" event={"ID":"3090f41d-918d-42ac-b7ae-837a2cb25685","Type":"ContainerStarted","Data":"9f8ca0b65b4f313da24f0cf1ce80193b3b58c66fe801261f3bc45c2b2cf3dfc8"} Jul 08 16:57:54.200293 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.199287 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:54.202401 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.202370 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" event={"ID":"5cd01afb-1f34-4bc0-b447-715f5df04076","Type":"ContainerStarted","Data":"490022063d17c9de54e2ecba44da80301c2531f9ae669991e7358546e8a251b4"} Jul 08 16:57:54.205208 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.204953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"914a0af8389e9d5da5df9a1d7468637e2cde5fddcfbc562d1bb23730938edd0d"} Jul 08 16:57:54.210273 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.209632 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd5d8cc-5r9lr" event={"ID":"165b7775-49ae-4aa6-85de-8a4668a958a5","Type":"ContainerStarted","Data":"f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec"} Jul 08 16:57:54.213754 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.213641 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-zjlt6" Jul 08 16:57:54.279424 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.278721 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-zjlt6" podStartSLOduration=1.770793978 podStartE2EDuration="22.278695719s" totalImagesPullingTime="20.507901741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:54.276422276 +0000 UTC m=+86.039414136" watchObservedRunningTime="2026-07-08 16:57:54.278695719 +0000 UTC m=+86.041687583" Jul 08 16:57:54.341367 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.341312 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-697fd5d8cc-5r9lr" podStartSLOduration=4.243673761 podStartE2EDuration="15.341289959s" totalImagesPullingTime="11.097616198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:54.341017034 +0000 UTC m=+86.104008895" watchObservedRunningTime="2026-07-08 16:57:54.341289959 +0000 UTC m=+86.104281820" Jul 08 16:57:54.482965 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.482840 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:54.619056 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:54.618972 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:55.215622 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:55.215394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"367afe908dab98f1f7bb09c43a6e97e5c03ae9b03290f868756b7b3cbbba6de0"} Jul 08 16:57:55.675589 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:55.675521 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697fd5d8cc-5r9lr"] Jul 08 16:57:57.225020 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.224922 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d" exitCode=0 Jul 08 16:57:57.225498 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.225017 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d"} Jul 08 16:57:57.226974 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.226915 2563 generic.go:356] "Generic (PLEG): container finished" podID="3dbf3d66-3020-4891-af4b-06f1b4b072d8" containerID="d6222b50b8881f08025713fcf6dda7f8504f69e0276dbe4ccadd15427cbdc976" exitCode=0 Jul 08 16:57:57.227041 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.226983 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smmh6" event={"ID":"3dbf3d66-3020-4891-af4b-06f1b4b072d8","Type":"ContainerDied","Data":"d6222b50b8881f08025713fcf6dda7f8504f69e0276dbe4ccadd15427cbdc976"} Jul 08 16:57:57.229797 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.229768 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" event={"ID":"5cd01afb-1f34-4bc0-b447-715f5df04076","Type":"ContainerStarted","Data":"bb335be9d9cdade6188fa8be97250b2b0f44b9fbe16023a6e8f26bcb850cd13c"} Jul 08 16:57:57.229890 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.229835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" event={"ID":"5cd01afb-1f34-4bc0-b447-715f5df04076","Type":"ContainerStarted","Data":"72b686d4c5e42cbb3954171322cc6b81f9ebae30792355fefa82c36093a2fe15"} Jul 08 16:57:57.229890 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.229872 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" event={"ID":"5cd01afb-1f34-4bc0-b447-715f5df04076","Type":"ContainerStarted","Data":"f500ce5261659ef138555b5cbcd8ca3d2cae7e390c548232a0a8541879afeac4"} Jul 08 16:57:57.231553 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.231529 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" exitCode=0 Jul 08 16:57:57.231645 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.231559 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} Jul 08 16:57:57.404453 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:57.404387 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78d56bcd6c-qlhrf" podStartSLOduration=4.38861073 podStartE2EDuration="7.404370339s" totalImagesPullingTime="3.015759609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:57.403884609 +0000 UTC m=+89.166876482" watchObservedRunningTime="2026-07-08 16:57:57.404370339 +0000 UTC m=+89.167362199" Jul 08 16:57:58.239993 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:58.239932 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smmh6" event={"ID":"3dbf3d66-3020-4891-af4b-06f1b4b072d8","Type":"ContainerStarted","Data":"2e15a19900626e6cdf1b55e1e07370cd7d9bc9f1202452af2369db446c00e140"} Jul 08 16:57:58.240668 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:58.240215 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smmh6" event={"ID":"3dbf3d66-3020-4891-af4b-06f1b4b072d8","Type":"ContainerStarted","Data":"8ab3c7bf230d8625adab8ebea1a9f5a0dd6f05882399c30816239dee34fca7d1"} Jul 08 16:57:58.307726 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:58.304414 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-smmh6" podStartSLOduration=10.001875659 podStartE2EDuration="13.304393438s" totalImagesPullingTime="3.197298706s" totalInitContainerRuntime="105.219073ms" isStatefulPod=true podCreationTimestamp="2026-07-08 16:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:58.301244973 +0000 UTC m=+90.064236836" watchObservedRunningTime="2026-07-08 16:57:58.304393438 +0000 UTC m=+90.067385301" Jul 08 16:57:58.954096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:58.954041 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:59.210276 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:59.210178 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:59.541005 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:59.540969 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:59.691300 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:59.691262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:57:59.796503 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:57:59.796336 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:58:00.867076 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:00.867041 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:01.031083 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:01.031049 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:01.476011 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:01.475973 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:01.641139 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:01.641102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:02.262046 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:02.262008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} Jul 08 16:58:02.264669 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:02.264648 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837"} Jul 08 16:58:02.264787 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:02.264715 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc"} Jul 08 16:58:03.271658 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:03.271618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445"} Jul 08 16:58:03.272189 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:03.271701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6"} Jul 08 16:58:03.272189 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:03.271735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0"} Jul 08 16:58:03.273520 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:03.273500 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} Jul 08 16:58:04.280213 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:04.280180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerStarted","Data":"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41"} Jul 08 16:58:04.319563 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:04.319507 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.574207333 podStartE2EDuration="18.319488282s" totalImagesPullingTime="8.722316262s" totalInitContainerRuntime="22.964687ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:58:04.31602636 +0000 UTC m=+96.079018223" watchObservedRunningTime="2026-07-08 16:58:04.319488282 +0000 UTC m=+96.082480141" Jul 08 16:58:07.295501 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.295465 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} Jul 08 16:58:07.295932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.295551 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} Jul 08 16:58:07.295932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.295581 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} Jul 08 16:58:07.295932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.295606 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerStarted","Data":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} Jul 08 16:58:07.346943 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.346893 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.149603571 podStartE2EDuration="14.346875402s" totalImagesPullingTime="11.167849176s" totalInitContainerRuntime="29.422655ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:57:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:58:07.345513998 +0000 UTC m=+99.108505858" watchObservedRunningTime="2026-07-08 16:58:07.346875402 +0000 UTC m=+99.109867261" Jul 08 16:58:07.503023 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.502978 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:07.607941 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:07.607857 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:08.341334 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:08.341301 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:08.878841 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:08.878807 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:58:11.617827 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:11.617789 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:11.719807 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:11.719767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:11.820747 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:11.820707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:12.684834 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:12.684805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brkmd_dd0ce6a4-e00e-4f12-aba0-5524ccbac037/serve-healthcheck-canary/0.log" Jul 08 16:58:13.151597 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:13.151567 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pb29q" Jul 08 16:58:14.453132 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:14.453093 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:14.552405 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:14.552370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:14.636196 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:14.636135 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:21.242785 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.242715 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-697fd5d8cc-5r9lr" podUID="165b7775-49ae-4aa6-85de-8a4668a958a5" containerName="console" containerID="cri-o://f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec" gracePeriod=15 Jul 08 16:58:21.541842 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.541812 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:58:21.669455 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669416 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-serving-cert\") pod \"165b7775-49ae-4aa6-85de-8a4668a958a5\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " Jul 08 16:58:21.669638 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669490 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-oauth-serving-cert\") pod \"165b7775-49ae-4aa6-85de-8a4668a958a5\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " Jul 08 16:58:21.669638 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669529 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-service-ca\") pod \"165b7775-49ae-4aa6-85de-8a4668a958a5\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " Jul 08 16:58:21.669638 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669581 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktzqm\" (UniqueName: \"kubernetes.io/projected/165b7775-49ae-4aa6-85de-8a4668a958a5-kube-api-access-ktzqm\") pod \"165b7775-49ae-4aa6-85de-8a4668a958a5\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " Jul 08 16:58:21.669638 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669615 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-oauth-config\") pod \"165b7775-49ae-4aa6-85de-8a4668a958a5\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " Jul 08 16:58:21.669850 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669647 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-console-config\") pod \"165b7775-49ae-4aa6-85de-8a4668a958a5\" (UID: \"165b7775-49ae-4aa6-85de-8a4668a958a5\") " Jul 08 16:58:21.670030 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.669953 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-service-ca" (OuterVolumeSpecName: "service-ca") pod "165b7775-49ae-4aa6-85de-8a4668a958a5" (UID: "165b7775-49ae-4aa6-85de-8a4668a958a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:58:21.670113 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.670049 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "165b7775-49ae-4aa6-85de-8a4668a958a5" (UID: "165b7775-49ae-4aa6-85de-8a4668a958a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:58:21.670189 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.670108 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-console-config" (OuterVolumeSpecName: "console-config") pod "165b7775-49ae-4aa6-85de-8a4668a958a5" (UID: "165b7775-49ae-4aa6-85de-8a4668a958a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:58:21.671949 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.671922 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165b7775-49ae-4aa6-85de-8a4668a958a5-kube-api-access-ktzqm" (OuterVolumeSpecName: "kube-api-access-ktzqm") pod "165b7775-49ae-4aa6-85de-8a4668a958a5" (UID: "165b7775-49ae-4aa6-85de-8a4668a958a5"). InnerVolumeSpecName "kube-api-access-ktzqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:58:21.672228 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.671927 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "165b7775-49ae-4aa6-85de-8a4668a958a5" (UID: "165b7775-49ae-4aa6-85de-8a4668a958a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:58:21.672228 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.671987 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "165b7775-49ae-4aa6-85de-8a4668a958a5" (UID: "165b7775-49ae-4aa6-85de-8a4668a958a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:58:21.770355 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.770322 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-oauth-serving-cert\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:58:21.770355 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.770351 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-service-ca\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:58:21.770355 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.770360 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktzqm\" (UniqueName: \"kubernetes.io/projected/165b7775-49ae-4aa6-85de-8a4668a958a5-kube-api-access-ktzqm\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:58:21.770580 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.770369 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-oauth-config\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:58:21.770580 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.770378 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/165b7775-49ae-4aa6-85de-8a4668a958a5-console-config\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:58:21.770580 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:21.770386 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/165b7775-49ae-4aa6-85de-8a4668a958a5-console-serving-cert\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:58:22.350881 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.350846 2563 generic.go:356] "Generic (PLEG): container finished" podID="165b7775-49ae-4aa6-85de-8a4668a958a5" containerID="f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec" exitCode=0 Jul 08 16:58:22.351492 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.350887 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd5d8cc-5r9lr" Jul 08 16:58:22.351492 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.350933 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd5d8cc-5r9lr" event={"ID":"165b7775-49ae-4aa6-85de-8a4668a958a5","Type":"ContainerDied","Data":"f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec"} Jul 08 16:58:22.351492 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.350993 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd5d8cc-5r9lr" event={"ID":"165b7775-49ae-4aa6-85de-8a4668a958a5","Type":"ContainerDied","Data":"cf2bf9c9a5b5a9557f42e0945f228f4bb4d582048d76b3befa390398b1565484"} Jul 08 16:58:22.351492 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.351021 2563 scope.go:117] "RemoveContainer" containerID="f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec" Jul 08 16:58:22.359725 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.359707 2563 scope.go:117] "RemoveContainer" containerID="f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec" Jul 08 16:58:22.360052 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:58:22.360031 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec\": container with ID starting with f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec not found: ID does not exist" containerID="f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec" Jul 08 16:58:22.360112 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.360062 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec"} err="failed to get container status \"f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec\": rpc error: code = NotFound desc = could not find container \"f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec\": container with ID starting with f10ac0ab6ffbfe0cc37d9383ec747c7055f5c6cb6b3078794dfd9a9b1ab549ec not found: ID does not exist" Jul 08 16:58:22.374930 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.374893 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697fd5d8cc-5r9lr"] Jul 08 16:58:22.379425 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.379394 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-697fd5d8cc-5r9lr"] Jul 08 16:58:22.480580 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.480541 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:22.559567 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.559535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:22.795086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:22.795057 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165b7775-49ae-4aa6-85de-8a4668a958a5" path="/var/lib/kubelet/pods/165b7775-49ae-4aa6-85de-8a4668a958a5/volumes" Jul 08 16:58:24.977365 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:24.977332 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:58:53.879082 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:53.879042 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:58:53.897932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:53.897904 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:58:54.486605 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:58:54.486577 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:02.916529 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:02.916489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:02.992718 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:02.992681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:07.297736 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.297698 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:59:07.519504 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.519462 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="alertmanager" containerID="cri-o://6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc" gracePeriod=120 Jul 08 16:59:07.519504 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.519511 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-metric" containerID="cri-o://5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445" gracePeriod=120 Jul 08 16:59:07.519767 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.519538 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy" containerID="cri-o://d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6" gracePeriod=120 Jul 08 16:59:07.519767 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.519542 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-web" containerID="cri-o://c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0" gracePeriod=120 Jul 08 16:59:07.519767 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.519577 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="prom-label-proxy" containerID="cri-o://b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41" gracePeriod=120 Jul 08 16:59:07.519767 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:07.519580 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="config-reloader" containerID="cri-o://2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837" gracePeriod=120 Jul 08 16:59:08.526120 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526089 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41" exitCode=0 Jul 08 16:59:08.526120 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526112 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6" exitCode=0 Jul 08 16:59:08.526120 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526118 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837" exitCode=0 Jul 08 16:59:08.526120 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526124 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc" exitCode=0 Jul 08 16:59:08.526657 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526170 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41"} Jul 08 16:59:08.526657 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526235 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6"} Jul 08 16:59:08.526657 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526261 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837"} Jul 08 16:59:08.526657 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.526289 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc"} Jul 08 16:59:08.782436 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.782363 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:08.856917 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.856879 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-web\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.856924 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-cluster-tls-config\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.856956 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-web-config\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.856997 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-main-db\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857048 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857086 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857083 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-out\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857110 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-trusted-ca-bundle\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857169 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-metrics-client-ca\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857200 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-volume\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857232 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857259 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-main-tls\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857288 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-tls-assets\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857382 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857346 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqgwv\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-kube-api-access-dqgwv\") pod \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\" (UID: \"2ba51c63-7b7f-40de-9127-21f27e86c7e9\") " Jul 08 16:59:08.857729 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857410 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:59:08.857729 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857569 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:08.857729 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857629 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-main-db\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.857729 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857650 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-metrics-client-ca\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.857932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.857882 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:08.860185 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.860105 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.861099 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.861071 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-volume" (OuterVolumeSpecName: "config-volume") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.861262 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.861204 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-kube-api-access-dqgwv" (OuterVolumeSpecName: "kube-api-access-dqgwv") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "kube-api-access-dqgwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:59:08.861262 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.861220 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.861262 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.861240 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.861783 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.861758 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-out" (OuterVolumeSpecName: "config-out") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:59:08.862176 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.862138 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:59:08.862992 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.862970 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.864416 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.864387 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.872082 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.872058 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-web-config" (OuterVolumeSpecName: "web-config") pod "2ba51c63-7b7f-40de-9127-21f27e86c7e9" (UID: "2ba51c63-7b7f-40de-9127-21f27e86c7e9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:08.958091 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958055 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958091 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958094 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-out\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958113 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ba51c63-7b7f-40de-9127-21f27e86c7e9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958129 2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-config-volume\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958172 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958187 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-main-tls\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958200 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-tls-assets\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958213 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqgwv\" (UniqueName: \"kubernetes.io/projected/2ba51c63-7b7f-40de-9127-21f27e86c7e9-kube-api-access-dqgwv\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958225 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958237 2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-cluster-tls-config\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:08.958310 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:08.958269 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ba51c63-7b7f-40de-9127-21f27e86c7e9-web-config\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:09.533034 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533000 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445" exitCode=0 Jul 08 16:59:09.533034 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533031 2563 generic.go:356] "Generic (PLEG): container finished" podID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerID="c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0" exitCode=0 Jul 08 16:59:09.533517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533070 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445"} Jul 08 16:59:09.533517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533091 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.533517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0"} Jul 08 16:59:09.533517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533205 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ba51c63-7b7f-40de-9127-21f27e86c7e9","Type":"ContainerDied","Data":"914a0af8389e9d5da5df9a1d7468637e2cde5fddcfbc562d1bb23730938edd0d"} Jul 08 16:59:09.533517 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.533232 2563 scope.go:117] "RemoveContainer" containerID="b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41" Jul 08 16:59:09.541119 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.541102 2563 scope.go:117] "RemoveContainer" containerID="5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445" Jul 08 16:59:09.547866 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.547848 2563 scope.go:117] "RemoveContainer" containerID="d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6" Jul 08 16:59:09.554434 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.554416 2563 scope.go:117] "RemoveContainer" containerID="c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0" Jul 08 16:59:09.558132 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.558113 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:59:09.561819 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.561799 2563 scope.go:117] "RemoveContainer" containerID="2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837" Jul 08 16:59:09.566835 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.566805 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:59:09.569124 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.569103 2563 scope.go:117] "RemoveContainer" containerID="6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc" Jul 08 16:59:09.575980 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.575965 2563 scope.go:117] "RemoveContainer" containerID="13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d" Jul 08 16:59:09.582628 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.582604 2563 scope.go:117] "RemoveContainer" containerID="b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41" Jul 08 16:59:09.582873 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.582853 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41\": container with ID starting with b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41 not found: ID does not exist" containerID="b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41" Jul 08 16:59:09.582956 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.582887 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41"} err="failed to get container status \"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41\": rpc error: code = NotFound desc = could not find container \"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41\": container with ID starting with b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41 not found: ID does not exist" Jul 08 16:59:09.582956 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.582916 2563 scope.go:117] "RemoveContainer" containerID="5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445" Jul 08 16:59:09.583178 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.583141 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445\": container with ID starting with 5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445 not found: ID does not exist" containerID="5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445" Jul 08 16:59:09.583224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583184 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445"} err="failed to get container status \"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445\": rpc error: code = NotFound desc = could not find container \"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445\": container with ID starting with 5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445 not found: ID does not exist" Jul 08 16:59:09.583224 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583200 2563 scope.go:117] "RemoveContainer" containerID="d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6" Jul 08 16:59:09.583433 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.583415 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6\": container with ID starting with d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6 not found: ID does not exist" containerID="d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6" Jul 08 16:59:09.583475 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583437 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6"} err="failed to get container status \"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6\": rpc error: code = NotFound desc = could not find container \"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6\": container with ID starting with d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6 not found: ID does not exist" Jul 08 16:59:09.583475 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583452 2563 scope.go:117] "RemoveContainer" containerID="c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0" Jul 08 16:59:09.583637 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.583622 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0\": container with ID starting with c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0 not found: ID does not exist" containerID="c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0" Jul 08 16:59:09.583677 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583639 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0"} err="failed to get container status \"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0\": rpc error: code = NotFound desc = could not find container \"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0\": container with ID starting with c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0 not found: ID does not exist" Jul 08 16:59:09.583677 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583651 2563 scope.go:117] "RemoveContainer" containerID="2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837" Jul 08 16:59:09.583822 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.583806 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837\": container with ID starting with 2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837 not found: ID does not exist" containerID="2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837" Jul 08 16:59:09.583862 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583827 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837"} err="failed to get container status \"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837\": rpc error: code = NotFound desc = could not find container \"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837\": container with ID starting with 2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837 not found: ID does not exist" Jul 08 16:59:09.583862 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.583842 2563 scope.go:117] "RemoveContainer" containerID="6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc" Jul 08 16:59:09.584026 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.584012 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc\": container with ID starting with 6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc not found: ID does not exist" containerID="6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc" Jul 08 16:59:09.584062 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584028 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc"} err="failed to get container status \"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc\": rpc error: code = NotFound desc = could not find container \"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc\": container with ID starting with 6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc not found: ID does not exist" Jul 08 16:59:09.584062 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584041 2563 scope.go:117] "RemoveContainer" containerID="13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d" Jul 08 16:59:09.584288 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:09.584271 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d\": container with ID starting with 13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d not found: ID does not exist" containerID="13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d" Jul 08 16:59:09.584361 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584295 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d"} err="failed to get container status \"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d\": rpc error: code = NotFound desc = could not find container \"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d\": container with ID starting with 13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d not found: ID does not exist" Jul 08 16:59:09.584361 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584316 2563 scope.go:117] "RemoveContainer" containerID="b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41" Jul 08 16:59:09.584523 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584506 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41"} err="failed to get container status \"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41\": rpc error: code = NotFound desc = could not find container \"b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41\": container with ID starting with b4f7c73a1c6e8b6bba9d3aaf64e4b5d9974fccdd4193f55124703190023e1e41 not found: ID does not exist" Jul 08 16:59:09.584568 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584524 2563 scope.go:117] "RemoveContainer" containerID="5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445" Jul 08 16:59:09.584698 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584681 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445"} err="failed to get container status \"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445\": rpc error: code = NotFound desc = could not find container \"5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445\": container with ID starting with 5bc42dd26f065ce42b926dd913b2c3edd154b27e1c059a0255aaa46afb744445 not found: ID does not exist" Jul 08 16:59:09.584757 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584699 2563 scope.go:117] "RemoveContainer" containerID="d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6" Jul 08 16:59:09.584890 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584875 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6"} err="failed to get container status \"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6\": rpc error: code = NotFound desc = could not find container \"d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6\": container with ID starting with d871485982e88fa07d0b444b2d2a7c23fbacc83be84896550e895590fcce44d6 not found: ID does not exist" Jul 08 16:59:09.584935 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.584891 2563 scope.go:117] "RemoveContainer" containerID="c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0" Jul 08 16:59:09.585052 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585038 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0"} err="failed to get container status \"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0\": rpc error: code = NotFound desc = could not find container \"c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0\": container with ID starting with c60ef33e5d90c7c3f44ded432ab31936a3fcce21c43b61306c34ffcfb61d0bf0 not found: ID does not exist" Jul 08 16:59:09.585052 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585052 2563 scope.go:117] "RemoveContainer" containerID="2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837" Jul 08 16:59:09.585247 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585231 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837"} err="failed to get container status \"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837\": rpc error: code = NotFound desc = could not find container \"2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837\": container with ID starting with 2607976fdc035f23398b2fcba537656badf80de74c880bfd677846890f877837 not found: ID does not exist" Jul 08 16:59:09.585303 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585247 2563 scope.go:117] "RemoveContainer" containerID="6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc" Jul 08 16:59:09.585444 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585428 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc"} err="failed to get container status \"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc\": rpc error: code = NotFound desc = could not find container \"6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc\": container with ID starting with 6401cb3e40667fc6506907eafce29439fe27e1a5dfb829814c53bf17cf9aecbc not found: ID does not exist" Jul 08 16:59:09.585487 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585444 2563 scope.go:117] "RemoveContainer" containerID="13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d" Jul 08 16:59:09.585616 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.585602 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d"} err="failed to get container status \"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d\": rpc error: code = NotFound desc = could not find container \"13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d\": container with ID starting with 13b7fd2a30fcfb520b26514419eca10c7f8a731aa248132e33c30ffa1edb571d not found: ID does not exist" Jul 08 16:59:09.605285 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605257 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:59:09.605522 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605510 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165b7775-49ae-4aa6-85de-8a4668a958a5" containerName="console" Jul 08 16:59:09.605571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605524 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="165b7775-49ae-4aa6-85de-8a4668a958a5" containerName="console" Jul 08 16:59:09.605571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605533 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-web" Jul 08 16:59:09.605571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605546 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-web" Jul 08 16:59:09.605571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605556 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="config-reloader" Jul 08 16:59:09.605571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605562 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="config-reloader" Jul 08 16:59:09.605571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605570 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="init-config-reloader" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605577 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="init-config-reloader" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605582 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-metric" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605587 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-metric" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605597 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="alertmanager" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605602 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="alertmanager" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605609 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605614 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605628 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="prom-label-proxy" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605632 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="prom-label-proxy" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605680 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605687 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="165b7775-49ae-4aa6-85de-8a4668a958a5" containerName="console" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605693 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="config-reloader" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605699 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-metric" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605705 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="alertmanager" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605710 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="prom-label-proxy" Jul 08 16:59:09.605742 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.605715 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" containerName="kube-rbac-proxy-web" Jul 08 16:59:09.609182 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.609167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.615964 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.615947 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 16:59:09.616207 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.616191 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 16:59:09.616289 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.616272 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 16:59:09.616485 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.616465 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 16:59:09.616571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.616518 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dchxq\"" Jul 08 16:59:09.616571 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.616540 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 16:59:09.617210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.617193 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 16:59:09.617268 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.617218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 16:59:09.619107 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.619094 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 16:59:09.626398 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.626378 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 16:59:09.639209 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.639176 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:59:09.662613 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662773 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662625 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgfj2\" (UniqueName: \"kubernetes.io/projected/6c8e8d78-7829-499d-ad86-cc79d27e2d40-kube-api-access-tgfj2\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662773 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c8e8d78-7829-499d-ad86-cc79d27e2d40-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662773 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662920 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662777 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c8e8d78-7829-499d-ad86-cc79d27e2d40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662920 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662806 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c8e8d78-7829-499d-ad86-cc79d27e2d40-config-out\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662920 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c8e8d78-7829-499d-ad86-cc79d27e2d40-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.662920 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662900 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.663064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662938 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.663064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.663064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.662989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.663064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.663021 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-web-config\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.663064 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.663055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c8e8d78-7829-499d-ad86-cc79d27e2d40-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.763757 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.763757 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763761 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgfj2\" (UniqueName: \"kubernetes.io/projected/6c8e8d78-7829-499d-ad86-cc79d27e2d40-kube-api-access-tgfj2\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763787 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c8e8d78-7829-499d-ad86-cc79d27e2d40-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763818 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763845 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c8e8d78-7829-499d-ad86-cc79d27e2d40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c8e8d78-7829-499d-ad86-cc79d27e2d40-config-out\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763898 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c8e8d78-7829-499d-ad86-cc79d27e2d40-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.763989 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764021 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.764013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764504 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.764048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-web-config\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764504 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.764092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c8e8d78-7829-499d-ad86-cc79d27e2d40-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.764504 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.764296 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c8e8d78-7829-499d-ad86-cc79d27e2d40-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.765026 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.764820 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c8e8d78-7829-499d-ad86-cc79d27e2d40-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.765026 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.764833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c8e8d78-7829-499d-ad86-cc79d27e2d40-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.766932 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.766903 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.767063 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.767040 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-web-config\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.767127 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.767061 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.767127 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.767111 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.767476 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.767446 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.767579 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.767528 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c8e8d78-7829-499d-ad86-cc79d27e2d40-config-out\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.767824 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.767806 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c8e8d78-7829-499d-ad86-cc79d27e2d40-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.768278 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.768261 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.768453 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.768434 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c8e8d78-7829-499d-ad86-cc79d27e2d40-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.774348 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.774327 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgfj2\" (UniqueName: \"kubernetes.io/projected/6c8e8d78-7829-499d-ad86-cc79d27e2d40-kube-api-access-tgfj2\") pod \"alertmanager-main-0\" (UID: \"6c8e8d78-7829-499d-ad86-cc79d27e2d40\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:09.918427 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:09.918339 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 16:59:10.064922 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:10.064885 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 16:59:10.068226 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:59:10.068196 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8e8d78_7829_499d_ad86_cc79d27e2d40.slice/crio-00f0dfd227be1d85413f30d3705fcbc1b7a9dad3f379c2887f6c79ea5d8397bc WatchSource:0}: Error finding container 00f0dfd227be1d85413f30d3705fcbc1b7a9dad3f379c2887f6c79ea5d8397bc: Status 404 returned error can't find the container with id 00f0dfd227be1d85413f30d3705fcbc1b7a9dad3f379c2887f6c79ea5d8397bc Jul 08 16:59:10.538395 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:10.538351 2563 generic.go:356] "Generic (PLEG): container finished" podID="6c8e8d78-7829-499d-ad86-cc79d27e2d40" containerID="14021a51bf49508bf2a4cb00074ec3ec18e0c5935f1da9e0e83a173d9d31e0c1" exitCode=0 Jul 08 16:59:10.538779 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:10.538416 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerDied","Data":"14021a51bf49508bf2a4cb00074ec3ec18e0c5935f1da9e0e83a173d9d31e0c1"} Jul 08 16:59:10.538779 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:10.538481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"00f0dfd227be1d85413f30d3705fcbc1b7a9dad3f379c2887f6c79ea5d8397bc"} Jul 08 16:59:10.796189 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:10.796136 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ba51c63-7b7f-40de-9127-21f27e86c7e9" path="/var/lib/kubelet/pods/2ba51c63-7b7f-40de-9127-21f27e86c7e9/volumes" Jul 08 16:59:11.527855 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.527818 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:59:11.528184 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.528135 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" gracePeriod=600 Jul 08 16:59:11.528274 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.528185 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-web" containerID="cri-o://ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" gracePeriod=600 Jul 08 16:59:11.528274 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.528198 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy" containerID="cri-o://2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" gracePeriod=600 Jul 08 16:59:11.528274 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.528214 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="thanos-sidecar" containerID="cri-o://f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" gracePeriod=600 Jul 08 16:59:11.528415 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.528121 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="prometheus" containerID="cri-o://088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" gracePeriod=600 Jul 08 16:59:11.528415 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.528194 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="config-reloader" containerID="cri-o://03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" gracePeriod=600 Jul 08 16:59:11.547047 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.547019 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"17a267930e4ca186ce56e608d33c0d05e2d0a35e71276810564729d2c7c564d0"} Jul 08 16:59:11.547419 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.547102 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"24b3a3555f1fc74c77e10fa8e4347e715ea35f1af591b95da9df8f58af7a064e"} Jul 08 16:59:11.547419 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.547174 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"07e823a2c7ae162e8097c60a65b0050c2b6af9c1e23caad30578ad3e6693ab20"} Jul 08 16:59:11.547419 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.547217 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"35f8dc1c3aef74ac52cf6232159cfbcc31089f552de9e852b1333611212214a5"} Jul 08 16:59:11.547419 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.547256 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"7756d071f85d2ee705b86a3224b12cc9ec77431753b28193dad41a301133a375"} Jul 08 16:59:11.547419 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.547298 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c8e8d78-7829-499d-ad86-cc79d27e2d40","Type":"ContainerStarted","Data":"b3ea7074548e47dab51e9b8fb90201a87de5bb643483360dee664dbac3084216"} Jul 08 16:59:11.619367 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.619065 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.603668369 podStartE2EDuration="2.619043361s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.374992ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:59:11.614436162 +0000 UTC m=+163.377428023" watchObservedRunningTime="2026-07-08 16:59:11.619043361 +0000 UTC m=+163.382035225" Jul 08 16:59:11.801270 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.801245 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:11.883434 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883399 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-serving-certs-ca-bundle\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883441 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-thanos-prometheus-http-client-file\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883477 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-kubelet-serving-ca-bundle\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883515 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-0\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883539 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-1\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883567 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-db\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883596 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-tls\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883654 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883623 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-metrics-client-certs\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883674 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883697 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-trusted-ca-bundle\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883734 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp9p6\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-kube-api-access-xp9p6\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883774 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-2\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883801 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-grpc-tls\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883851 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config-out\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883881 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-metrics-client-ca\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883906 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-tls-assets\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883929 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883939 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-kube-rbac-proxy\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.883995 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.883973 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.884551 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.884003 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-web-config\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.884551 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.884031 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\" (UID: \"44661a93-d8da-4daf-b0e4-4e9f22dd9e78\") " Jul 08 16:59:11.884551 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.884055 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.884551 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.884217 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.885161 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.884775 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.885161 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.884871 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.885772 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.885648 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.886861 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.886681 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.886965 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.886934 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-kube-api-access-xp9p6" (OuterVolumeSpecName: "kube-api-access-xp9p6") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "kube-api-access-xp9p6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887068 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887086 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887114 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887131 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887164 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887179 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887194 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.887234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.887208 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.888081 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.888044 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.888184 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.888115 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.888757 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.888727 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 16:59:11.888854 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.888766 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.889104 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.889073 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.889262 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.889115 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.889376 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.889339 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.889621 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.889596 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:59:11.889802 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.889777 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config-out" (OuterVolumeSpecName: "config-out") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 16:59:11.890846 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.890824 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config" (OuterVolumeSpecName: "config") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.899871 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.899846 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-web-config" (OuterVolumeSpecName: "web-config") pod "44661a93-d8da-4daf-b0e4-4e9f22dd9e78" (UID: "44661a93-d8da-4daf-b0e4-4e9f22dd9e78"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 16:59:11.988424 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988388 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-prometheus-k8s-db\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988424 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988418 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-metrics-client-certs\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988424 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988429 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988439 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp9p6\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-kube-api-access-xp9p6\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988449 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-grpc-tls\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988457 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-config-out\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988465 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-configmap-metrics-client-ca\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988475 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-tls-assets\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988483 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-kube-rbac-proxy\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988492 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988501 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-web-config\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988510 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:11.988633 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:11.988519 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44661a93-d8da-4daf-b0e4-4e9f22dd9e78-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 16:59:12.554160 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554115 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" exitCode=0 Jul 08 16:59:12.554160 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554139 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" exitCode=0 Jul 08 16:59:12.554160 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554161 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" exitCode=0 Jul 08 16:59:12.554160 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554169 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" exitCode=0 Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554175 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" exitCode=0 Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554180 2563 generic.go:356] "Generic (PLEG): container finished" podID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" exitCode=0 Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554210 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554202 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554438 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554495 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.554611 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554491 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} Jul 08 16:59:12.554919 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554615 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} Jul 08 16:59:12.554919 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} Jul 08 16:59:12.554919 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554702 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} Jul 08 16:59:12.554919 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.554745 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44661a93-d8da-4daf-b0e4-4e9f22dd9e78","Type":"ContainerDied","Data":"367afe908dab98f1f7bb09c43a6e97e5c03ae9b03290f868756b7b3cbbba6de0"} Jul 08 16:59:12.562346 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.562220 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.568956 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.568938 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.575429 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.575414 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.581701 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.581683 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.585912 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.585892 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:59:12.588844 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.588818 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.596167 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.596127 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.599462 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.599441 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:59:12.602929 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.602912 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.603222 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.603199 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.603265 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603237 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} err="failed to get container status \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" Jul 08 16:59:12.603265 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603260 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.603489 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.603472 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.603525 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603495 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} err="failed to get container status \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" Jul 08 16:59:12.603525 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603509 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.603738 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.603720 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.603786 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603739 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} err="failed to get container status \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" Jul 08 16:59:12.603786 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603751 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.603953 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.603937 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.603993 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603957 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} err="failed to get container status \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" Jul 08 16:59:12.603993 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.603969 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.604140 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.604126 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.604210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604157 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} err="failed to get container status \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" Jul 08 16:59:12.604210 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604169 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.604358 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.604340 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.604413 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604362 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} err="failed to get container status \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" Jul 08 16:59:12.604413 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604377 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.604612 ip-10-0-143-156 kubenswrapper[2563]: E0708 16:59:12.604595 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.604647 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604616 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} err="failed to get container status \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" Jul 08 16:59:12.604647 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604630 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.604860 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604842 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} err="failed to get container status \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" Jul 08 16:59:12.604896 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.604861 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.605077 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605055 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} err="failed to get container status \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" Jul 08 16:59:12.605077 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605076 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.605287 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605270 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} err="failed to get container status \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" Jul 08 16:59:12.605337 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605288 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.605473 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605456 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} err="failed to get container status \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" Jul 08 16:59:12.605510 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605473 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.605683 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605667 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} err="failed to get container status \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" Jul 08 16:59:12.605683 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605683 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.605880 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605863 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} err="failed to get container status \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" Jul 08 16:59:12.605921 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.605881 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.606094 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606078 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} err="failed to get container status \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" Jul 08 16:59:12.606132 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606095 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.606309 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606293 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} err="failed to get container status \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" Jul 08 16:59:12.606356 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606309 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.606475 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606461 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} err="failed to get container status \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" Jul 08 16:59:12.606522 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606475 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.606634 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606616 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} err="failed to get container status \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" Jul 08 16:59:12.606673 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606633 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.606800 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606782 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} err="failed to get container status \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" Jul 08 16:59:12.606838 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.606800 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.607022 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607007 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} err="failed to get container status \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" Jul 08 16:59:12.607059 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607022 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.607229 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607214 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} err="failed to get container status \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" Jul 08 16:59:12.607268 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607230 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.607407 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607391 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} err="failed to get container status \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" Jul 08 16:59:12.607448 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607408 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.607582 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607568 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} err="failed to get container status \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" Jul 08 16:59:12.607622 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607583 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.607771 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607755 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} err="failed to get container status \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" Jul 08 16:59:12.607820 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607770 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.607950 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607935 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} err="failed to get container status \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" Jul 08 16:59:12.607999 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.607950 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.608132 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608116 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} err="failed to get container status \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" Jul 08 16:59:12.608202 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608133 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.608381 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608366 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} err="failed to get container status \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" Jul 08 16:59:12.608458 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608382 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.608644 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608625 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} err="failed to get container status \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" Jul 08 16:59:12.608698 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608647 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.608892 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608874 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} err="failed to get container status \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" Jul 08 16:59:12.608940 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.608894 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.609133 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609115 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} err="failed to get container status \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" Jul 08 16:59:12.609216 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609133 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.609385 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609370 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} err="failed to get container status \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" Jul 08 16:59:12.609437 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609386 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.609617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609600 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} err="failed to get container status \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" Jul 08 16:59:12.609617 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609616 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.609815 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609799 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} err="failed to get container status \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" Jul 08 16:59:12.609815 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.609815 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.610050 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610034 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} err="failed to get container status \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" Jul 08 16:59:12.610091 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610050 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.610275 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610260 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} err="failed to get container status \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" Jul 08 16:59:12.610336 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610275 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.610478 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610458 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} err="failed to get container status \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" Jul 08 16:59:12.610519 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610479 2563 scope.go:117] "RemoveContainer" containerID="d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03" Jul 08 16:59:12.610714 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610697 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03"} err="failed to get container status \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": rpc error: code = NotFound desc = could not find container \"d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03\": container with ID starting with d7440c41e77eb157ce3244e2ace7a350e10ca3d08069ce7a1d00bf0be6de7e03 not found: ID does not exist" Jul 08 16:59:12.610776 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610717 2563 scope.go:117] "RemoveContainer" containerID="2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e" Jul 08 16:59:12.610962 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610943 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e"} err="failed to get container status \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": rpc error: code = NotFound desc = could not find container \"2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e\": container with ID starting with 2353e1625b29bfef4a5ccb0ff866628318d78b0ffd6dedc7f9cdf79712068b1e not found: ID does not exist" Jul 08 16:59:12.611040 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.610964 2563 scope.go:117] "RemoveContainer" containerID="ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9" Jul 08 16:59:12.611209 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611189 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9"} err="failed to get container status \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": rpc error: code = NotFound desc = could not find container \"ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9\": container with ID starting with ee95fcb240261731676d0fa260abf7ebbf244be9d96db700db79e00e91f585c9 not found: ID does not exist" Jul 08 16:59:12.611294 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611209 2563 scope.go:117] "RemoveContainer" containerID="f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb" Jul 08 16:59:12.611413 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611397 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb"} err="failed to get container status \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": rpc error: code = NotFound desc = could not find container \"f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb\": container with ID starting with f85b48ae7afae2015aab15ab04eba003b54a18eacea7386350d6dfc7e9d956eb not found: ID does not exist" Jul 08 16:59:12.611465 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611414 2563 scope.go:117] "RemoveContainer" containerID="03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7" Jul 08 16:59:12.611618 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611598 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7"} err="failed to get container status \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": rpc error: code = NotFound desc = could not find container \"03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7\": container with ID starting with 03347cf456820c3bca1fafa789b268b07f94c228988ea173d70399416181b1b7 not found: ID does not exist" Jul 08 16:59:12.611618 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611617 2563 scope.go:117] "RemoveContainer" containerID="088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728" Jul 08 16:59:12.611902 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611870 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728"} err="failed to get container status \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": rpc error: code = NotFound desc = could not find container \"088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728\": container with ID starting with 088e863e24b350d9a1b9c6cee0117fd3745ed1ebdaca7153fabb9d1ad246f728 not found: ID does not exist" Jul 08 16:59:12.611902 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.611892 2563 scope.go:117] "RemoveContainer" containerID="c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667" Jul 08 16:59:12.612197 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.612170 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667"} err="failed to get container status \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": rpc error: code = NotFound desc = could not find container \"c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667\": container with ID starting with c244ce14cdb001075b6d95d3d697c656473024992b749e8faeda593dfa16a667 not found: ID does not exist" Jul 08 16:59:12.641939 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.641913 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:59:12.642260 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642244 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642261 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642274 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-thanos" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642282 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-thanos" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642296 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="thanos-sidecar" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642306 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="thanos-sidecar" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642320 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-web" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642328 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-web" Jul 08 16:59:12.642342 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642339 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="prometheus" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642347 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="prometheus" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642360 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="init-config-reloader" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642369 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="init-config-reloader" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642380 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="config-reloader" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642388 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="config-reloader" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642454 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-thanos" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642468 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="config-reloader" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642477 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="thanos-sidecar" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642486 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy-web" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642497 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="kube-rbac-proxy" Jul 08 16:59:12.642756 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.642503 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" containerName="prometheus" Jul 08 16:59:12.646649 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.646631 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.654062 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654040 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 16:59:12.654178 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654079 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 16:59:12.654232 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654222 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 16:59:12.654325 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654311 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 16:59:12.654445 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654429 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 16:59:12.654719 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654706 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8jvfnuh9jo1tb\"" Jul 08 16:59:12.654871 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.654836 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 16:59:12.655250 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.655236 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 16:59:12.655354 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.655338 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 16:59:12.656279 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.656264 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 16:59:12.656316 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.656264 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 16:59:12.656316 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.656297 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xgrmw\"" Jul 08 16:59:12.658211 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.658195 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 16:59:12.658561 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.658547 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 16:59:12.658641 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.658552 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 16:59:12.660830 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.660812 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 16:59:12.663754 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.663739 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 16:59:12.677771 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.677745 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:59:12.694481 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-config-out\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694603 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694490 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694603 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694603 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-web-config\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694758 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694613 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694758 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694655 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694758 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-config\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694758 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694758 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694743 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694912 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694768 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694912 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694912 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694854 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694912 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694883 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.694912 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694901 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.695096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.695096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.695096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.694984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.695096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.695017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.695096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.695050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2sgw\" (UniqueName: \"kubernetes.io/projected/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-kube-api-access-r2sgw\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.695096 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.695083 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.795923 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.795884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-config-out\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796102 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.795932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796102 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.795952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796102 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.795978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-web-config\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796102 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796102 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-config\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796265 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796205 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44661a93-d8da-4daf-b0e4-4e9f22dd9e78" path="/var/lib/kubelet/pods/44661a93-d8da-4daf-b0e4-4e9f22dd9e78/volumes" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796324 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796399 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796421 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2sgw\" (UniqueName: \"kubernetes.io/projected/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-kube-api-access-r2sgw\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.796809 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.797166 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.796818 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.797409 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.797322 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.798704 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.797885 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.798704 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.798115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.798704 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.798435 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.798704 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.798662 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.798704 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.798670 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.799497 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.799457 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.799497 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.799483 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801119 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801091 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-web-config\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801292 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-config-out\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801400 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801385 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801460 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801423 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801847 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801826 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801923 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801860 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801923 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801904 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.801993 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.801944 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.802234 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.802216 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-config\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.802439 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.802423 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.808499 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.808445 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2sgw\" (UniqueName: \"kubernetes.io/projected/71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c-kube-api-access-r2sgw\") pod \"prometheus-k8s-0\" (UID: \"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:12.955850 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:12.955806 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:13.100397 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:13.100312 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 16:59:13.106031 ip-10-0-143-156 kubenswrapper[2563]: W0708 16:59:13.106006 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ce3ab8_2941_4e75_98e3_9d9d6c4aa89c.slice/crio-4b45bb404eb7bda66977164ec302deda82f840e196427af8f51d31893c0712c8 WatchSource:0}: Error finding container 4b45bb404eb7bda66977164ec302deda82f840e196427af8f51d31893c0712c8: Status 404 returned error can't find the container with id 4b45bb404eb7bda66977164ec302deda82f840e196427af8f51d31893c0712c8 Jul 08 16:59:13.560284 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:13.560250 2563 generic.go:356] "Generic (PLEG): container finished" podID="71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c" containerID="237f3f3d3c0a284adaeb2be98e2fee136a544a3d590fa238da7fbfaf532b1438" exitCode=0 Jul 08 16:59:13.560639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:13.560310 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerDied","Data":"237f3f3d3c0a284adaeb2be98e2fee136a544a3d590fa238da7fbfaf532b1438"} Jul 08 16:59:13.560639 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:13.560364 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"4b45bb404eb7bda66977164ec302deda82f840e196427af8f51d31893c0712c8"} Jul 08 16:59:14.566731 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.566691 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"f0876e5df824754f0cd5f37b9c7d5f7bdb6ff68f9b35f06bb876a0bd047e4178"} Jul 08 16:59:14.567078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.566772 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"20a231285e5aaac64efde64c0194b3fa11f64c8a15309dd61cbfdb0515ef5d2a"} Jul 08 16:59:14.567078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.566813 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"060a4a13768dc35c4711ceb279d69dd9e62d6d9ff10e99a33233cf09a8e8e4be"} Jul 08 16:59:14.567078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.566855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"5e31eea61ef89976d64513d3c7510044763f5c3b04c2e11bd52b13f611dba790"} Jul 08 16:59:14.567078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.566892 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"4578b0309a79c0b566b368736c3a49c5e8aec26e0327ee0295240f1aa332139c"} Jul 08 16:59:14.567078 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.566926 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c","Type":"ContainerStarted","Data":"ce6cfabc22a17fcd9fec2ba7164c6d1639fa41efc3e98a9612dd11d7072fd46c"} Jul 08 16:59:14.604265 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.604203 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.586560796 podStartE2EDuration="2.60418373s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.622934ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:59:14.603436011 +0000 UTC m=+166.366427896" watchObservedRunningTime="2026-07-08 16:59:14.60418373 +0000 UTC m=+166.367175592" Jul 08 16:59:14.836060 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.835981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:14.920071 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:14.920037 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:17.955955 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:17.955920 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 16:59:22.506258 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:22.506142 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:22.648990 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:22.648957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:22.782691 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:22.782657 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:22.878968 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:22.878930 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:27.567459 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:27.567419 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:34.048325 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:34.048282 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:34.153484 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:34.153453 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:34.223528 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:34.223488 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:42.687315 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:42.687275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 16:59:42.785939 ip-10-0-143-156 kubenswrapper[2563]: I0708 16:59:42.785895 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:04.732902 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:04.732863 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:08.886913 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:08.886872 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:08.966495 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:08.966461 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:12.956655 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:12.956611 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:00:12.971846 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:12.971816 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:00:13.808008 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:13.807982 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:00:35.959493 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:35.959448 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:36.275628 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:36.275592 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:39.971774 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:39.971740 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:40.059192 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:40.059142 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:42.279509 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:42.279477 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:42.381791 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:42.381753 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:44.284998 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:44.284965 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:44.663884 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:44.663796 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:44.774122 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:44.774092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:44.840594 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:44.840562 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:50.623606 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:50.623526 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:50.722542 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:50.722509 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:00:52.899376 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:00:52.899334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:04.520982 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:04.520943 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:04.619732 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:04.619699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:14.908996 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:14.908953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:15.007472 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:15.007435 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:25.298937 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:25.298902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:26.782354 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:26.782313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:28.118456 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:28.118412 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:28.624910 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:28.624882 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 17:01:44.415335 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:44.415297 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:44.515966 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:44.515938 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:47.675114 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.675076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:47.693814 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.693780 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8"] Jul 08 17:01:47.696041 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.696020 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.698434 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.698398 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 17:01:47.698517 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.698497 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kubeflow-trainer-webhook-cert\"" Jul 08 17:01:47.699311 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.699290 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kubeflow-trainer-controller-manager-dockercfg-jg5t7\"" Jul 08 17:01:47.699403 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.699291 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kubeflow-trainer-config\"" Jul 08 17:01:47.699403 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.699339 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 17:01:47.710535 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.710514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8"] Jul 08 17:01:47.753139 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.753105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeflow-trainer-config\" (UniqueName: \"kubernetes.io/configmap/cfab7a27-1afb-4099-a14a-d6077ede175e-kubeflow-trainer-config\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.753324 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.753174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfab7a27-1afb-4099-a14a-d6077ede175e-cert\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.753324 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.753216 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w2rk\" (UniqueName: \"kubernetes.io/projected/cfab7a27-1afb-4099-a14a-d6077ede175e-kube-api-access-4w2rk\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.854634 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.854594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubeflow-trainer-config\" (UniqueName: \"kubernetes.io/configmap/cfab7a27-1afb-4099-a14a-d6077ede175e-kubeflow-trainer-config\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.854827 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.854644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfab7a27-1afb-4099-a14a-d6077ede175e-cert\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.854827 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.854670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2rk\" (UniqueName: \"kubernetes.io/projected/cfab7a27-1afb-4099-a14a-d6077ede175e-kube-api-access-4w2rk\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.855310 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.855288 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubeflow-trainer-config\" (UniqueName: \"kubernetes.io/configmap/cfab7a27-1afb-4099-a14a-d6077ede175e-kubeflow-trainer-config\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.857014 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.856990 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cfab7a27-1afb-4099-a14a-d6077ede175e-cert\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:47.859187 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.859140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:47.862961 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:47.862940 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w2rk\" (UniqueName: \"kubernetes.io/projected/cfab7a27-1afb-4099-a14a-d6077ede175e-kube-api-access-4w2rk\") pod \"kubeflow-trainer-controller-manager-86d75bcb49-wn4r8\" (UID: \"cfab7a27-1afb-4099-a14a-d6077ede175e\") " pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:48.001400 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:48.001315 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:01:48.004442 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:48.004424 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:48.131475 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:48.131454 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8"] Jul 08 17:01:48.133552 ip-10-0-143-156 kubenswrapper[2563]: W0708 17:01:48.133527 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfab7a27_1afb_4099_a14a_d6077ede175e.slice/crio-b0dfc20be02ef479bf9fd37f5e7f0563f24f11d018f9e5a04bef3bb7e93b4328 WatchSource:0}: Error finding container b0dfc20be02ef479bf9fd37f5e7f0563f24f11d018f9e5a04bef3bb7e93b4328: Status 404 returned error can't find the container with id b0dfc20be02ef479bf9fd37f5e7f0563f24f11d018f9e5a04bef3bb7e93b4328 Jul 08 17:01:48.135238 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:48.135222 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:01:49.137557 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:49.137524 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" event={"ID":"cfab7a27-1afb-4099-a14a-d6077ede175e","Type":"ContainerStarted","Data":"b0dfc20be02ef479bf9fd37f5e7f0563f24f11d018f9e5a04bef3bb7e93b4328"} Jul 08 17:01:51.148162 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:51.148117 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" event={"ID":"cfab7a27-1afb-4099-a14a-d6077ede175e","Type":"ContainerStarted","Data":"95cdfa115769ca55aae943895743295a6d6bac02ee5368089b07a4c0e27af67c"} Jul 08 17:01:51.148563 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:51.148259 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:01:51.167362 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:51.167308 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" podStartSLOduration=1.3776758230000001 podStartE2EDuration="4.167289921s" totalImagesPullingTime="2.789614098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:01:51.165659202 +0000 UTC m=+322.928651063" watchObservedRunningTime="2026-07-08 17:01:51.167289921 +0000 UTC m=+322.930281784" Jul 08 17:01:51.587824 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:01:51.587786 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:02:07.157104 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.157073 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kubeflow-trainer-controller-manager-86d75bcb49-wn4r8" Jul 08 17:02:07.591336 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.591295 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-npd78/must-gather-dbxht"] Jul 08 17:02:07.593702 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.593681 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.596132 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.596104 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-npd78\"/\"openshift-service-ca.crt\"" Jul 08 17:02:07.596300 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.596104 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-npd78\"/\"default-dockercfg-dzx2n\"" Jul 08 17:02:07.596463 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.596448 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-npd78\"/\"kube-root-ca.crt\"" Jul 08 17:02:07.604424 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.604399 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-npd78/must-gather-dbxht"] Jul 08 17:02:07.608782 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.608756 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:02:07.706273 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.706227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:02:07.720387 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.720354 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9lmf\" (UniqueName: \"kubernetes.io/projected/b7637fcd-c2d9-4075-825b-327a811fe8e1-kube-api-access-l9lmf\") pod \"must-gather-dbxht\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.720508 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.720407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7637fcd-c2d9-4075-825b-327a811fe8e1-must-gather-output\") pod \"must-gather-dbxht\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.821809 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.821770 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9lmf\" (UniqueName: \"kubernetes.io/projected/b7637fcd-c2d9-4075-825b-327a811fe8e1-kube-api-access-l9lmf\") pod \"must-gather-dbxht\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.821943 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.821822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7637fcd-c2d9-4075-825b-327a811fe8e1-must-gather-output\") pod \"must-gather-dbxht\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.822138 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.822123 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7637fcd-c2d9-4075-825b-327a811fe8e1-must-gather-output\") pod \"must-gather-dbxht\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.831186 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.831141 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9lmf\" (UniqueName: \"kubernetes.io/projected/b7637fcd-c2d9-4075-825b-327a811fe8e1-kube-api-access-l9lmf\") pod \"must-gather-dbxht\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:07.903102 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:07.903017 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:02:08.031917 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:08.031895 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-npd78/must-gather-dbxht"] Jul 08 17:02:08.034129 ip-10-0-143-156 kubenswrapper[2563]: W0708 17:02:08.034095 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7637fcd_c2d9_4075_825b_327a811fe8e1.slice/crio-a56ce17f5aa763cf8fa71d6c04e5fc5980a16f571a2494c80dc81559435b38a9 WatchSource:0}: Error finding container a56ce17f5aa763cf8fa71d6c04e5fc5980a16f571a2494c80dc81559435b38a9: Status 404 returned error can't find the container with id a56ce17f5aa763cf8fa71d6c04e5fc5980a16f571a2494c80dc81559435b38a9 Jul 08 17:02:08.210968 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:08.210884 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-npd78/must-gather-dbxht" event={"ID":"b7637fcd-c2d9-4075-825b-327a811fe8e1","Type":"ContainerStarted","Data":"a56ce17f5aa763cf8fa71d6c04e5fc5980a16f571a2494c80dc81559435b38a9"} Jul 08 17:02:14.246938 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:14.246890 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-npd78/must-gather-dbxht" event={"ID":"b7637fcd-c2d9-4075-825b-327a811fe8e1","Type":"ContainerStarted","Data":"ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae"} Jul 08 17:02:15.254214 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:15.254173 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-npd78/must-gather-dbxht" event={"ID":"b7637fcd-c2d9-4075-825b-327a811fe8e1","Type":"ContainerStarted","Data":"9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1"} Jul 08 17:02:15.272278 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:15.272216 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-npd78/must-gather-dbxht" podStartSLOduration=2.210256224 podStartE2EDuration="8.272201665s" totalImagesPullingTime="6.061945441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:02:15.270553843 +0000 UTC m=+347.033545704" watchObservedRunningTime="2026-07-08 17:02:15.272201665 +0000 UTC m=+347.035193524" Jul 08 17:02:17.652818 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:17.652782 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:02:17.727406 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:17.727357 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:02:24.059707 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:24.059615 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kubeflow-trainer-controller-manager-86d75bcb49-wn4r8_cfab7a27-1afb-4099-a14a-d6077ede175e/manager/0.log" Jul 08 17:02:24.543965 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:24.543920 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kubeflow-trainer-controller-manager-86d75bcb49-wn4r8_cfab7a27-1afb-4099-a14a-d6077ede175e/manager/0.log" Jul 08 17:02:25.047674 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:25.047641 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kubeflow-trainer-controller-manager-86d75bcb49-wn4r8_cfab7a27-1afb-4099-a14a-d6077ede175e/manager/0.log" Jul 08 17:02:54.698329 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:02:54.698292 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:03.453694 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:03.453660 2563 generic.go:356] "Generic (PLEG): container finished" podID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerID="ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae" exitCode=0 Jul 08 17:03:03.453694 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:03.453697 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-npd78/must-gather-dbxht" event={"ID":"b7637fcd-c2d9-4075-825b-327a811fe8e1","Type":"ContainerDied","Data":"ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae"} Jul 08 17:03:03.454182 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:03.453998 2563 scope.go:117] "RemoveContainer" containerID="ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae" Jul 08 17:03:03.507360 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:03.507328 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-npd78_must-gather-dbxht_b7637fcd-c2d9-4075-825b-327a811fe8e1/gather/0.log" Jul 08 17:03:03.985488 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:03.985447 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:04.150189 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:04.150134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:04.321202 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:04.321132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:07.152871 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:07.152835 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lgbxp_ec13312c-4802-428d-bb87-0a70978eb388/global-pull-secret-syncer/0.log" Jul 08 17:03:07.220131 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:07.220099 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" Jul 08 17:03:07.397413 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:07.397383 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-156.ec2.internal_0052804689573a175003f01e231a790e/haproxy/0.log" Jul 08 17:03:08.945602 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:08.945572 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-npd78/must-gather-dbxht"] Jul 08 17:03:08.946034 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:08.945744 2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-npd78/must-gather-dbxht" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="copy" containerID="cri-o://9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1" gracePeriod=2 Jul 08 17:03:08.953391 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:08.953369 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-npd78/must-gather-dbxht"] Jul 08 17:03:08.954691 ip-10-0-143-156 kubenswrapper[2563]: E0708 17:03:08.954654 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b7637fcd-c2d9-4075-825b-327a811fe8e1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-npd78\"/\"must-gather-dbxht\": pods \"must-gather-dbxht\" not found" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:03:09.180687 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.180662 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-npd78_must-gather-dbxht_b7637fcd-c2d9-4075-825b-327a811fe8e1/copy/0.log" Jul 08 17:03:09.182446 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.182426 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:03:09.184577 ip-10-0-143-156 kubenswrapper[2563]: E0708 17:03:09.184555 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-dbxht\" is forbidden: User \"system:node:ip-10-0-143-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-npd78\": no relationship found between node 'ip-10-0-143-156.ec2.internal' and this object" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:03:09.363684 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.363644 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9lmf\" (UniqueName: \"kubernetes.io/projected/b7637fcd-c2d9-4075-825b-327a811fe8e1-kube-api-access-l9lmf\") pod \"b7637fcd-c2d9-4075-825b-327a811fe8e1\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " Jul 08 17:03:09.363870 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.363741 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7637fcd-c2d9-4075-825b-327a811fe8e1-must-gather-output\") pod \"b7637fcd-c2d9-4075-825b-327a811fe8e1\" (UID: \"b7637fcd-c2d9-4075-825b-327a811fe8e1\") " Jul 08 17:03:09.365801 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.365774 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7637fcd-c2d9-4075-825b-327a811fe8e1-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "b7637fcd-c2d9-4075-825b-327a811fe8e1" (UID: "b7637fcd-c2d9-4075-825b-327a811fe8e1"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:03:09.365912 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.365889 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7637fcd-c2d9-4075-825b-327a811fe8e1-kube-api-access-l9lmf" (OuterVolumeSpecName: "kube-api-access-l9lmf") pod "b7637fcd-c2d9-4075-825b-327a811fe8e1" (UID: "b7637fcd-c2d9-4075-825b-327a811fe8e1"). InnerVolumeSpecName "kube-api-access-l9lmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:03:09.464603 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.464564 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9lmf\" (UniqueName: \"kubernetes.io/projected/b7637fcd-c2d9-4075-825b-327a811fe8e1-kube-api-access-l9lmf\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 17:03:09.464603 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.464596 2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7637fcd-c2d9-4075-825b-327a811fe8e1-must-gather-output\") on node \"ip-10-0-143-156.ec2.internal\" DevicePath \"\"" Jul 08 17:03:09.482685 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.482655 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-npd78_must-gather-dbxht_b7637fcd-c2d9-4075-825b-327a811fe8e1/copy/0.log" Jul 08 17:03:09.482997 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.482973 2563 generic.go:356] "Generic (PLEG): container finished" podID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerID="9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1" exitCode=143 Jul 08 17:03:09.483044 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.483024 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:03:09.483124 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.483062 2563 scope.go:117] "RemoveContainer" containerID="9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1" Jul 08 17:03:09.485418 ip-10-0-143-156 kubenswrapper[2563]: E0708 17:03:09.485395 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-dbxht\" is forbidden: User \"system:node:ip-10-0-143-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-npd78\": no relationship found between node 'ip-10-0-143-156.ec2.internal' and this object" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:03:09.490770 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.490749 2563 scope.go:117] "RemoveContainer" containerID="ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae" Jul 08 17:03:09.493109 ip-10-0-143-156 kubenswrapper[2563]: E0708 17:03:09.493083 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-dbxht\" is forbidden: User \"system:node:ip-10-0-143-156.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-npd78\": no relationship found between node 'ip-10-0-143-156.ec2.internal' and this object" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" pod="openshift-must-gather-npd78/must-gather-dbxht" Jul 08 17:03:09.501518 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.501498 2563 scope.go:117] "RemoveContainer" containerID="9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1" Jul 08 17:03:09.501820 ip-10-0-143-156 kubenswrapper[2563]: E0708 17:03:09.501802 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1\": container with ID starting with 9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1 not found: ID does not exist" containerID="9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1" Jul 08 17:03:09.501880 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.501828 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1"} err="failed to get container status \"9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1\": rpc error: code = NotFound desc = could not find container \"9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1\": container with ID starting with 9dfe77414b56921923548c1486f0a9fee74e46e160945a199e8a2f69089b37f1 not found: ID does not exist" Jul 08 17:03:09.501880 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.501846 2563 scope.go:117] "RemoveContainer" containerID="ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae" Jul 08 17:03:09.502113 ip-10-0-143-156 kubenswrapper[2563]: E0708 17:03:09.502092 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae\": container with ID starting with ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae not found: ID does not exist" containerID="ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae" Jul 08 17:03:09.502187 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:09.502118 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae"} err="failed to get container status \"ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae\": rpc error: code = NotFound desc = could not find container \"ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae\": container with ID starting with ab8e8f1e7e1f8e87e2bf739c7be62942fe39714a9a35a34c3fc8d59d4feef8ae not found: ID does not exist" Jul 08 17:03:10.075537 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:10.075500 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:10.165862 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:10.165827 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:10.800925 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:10.800893 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" path="/var/lib/kubelet/pods/b7637fcd-c2d9-4075-825b-327a811fe8e1/volumes" Jul 08 17:03:10.903970 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:10.903894 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/alertmanager/0.log" Jul 08 17:03:10.941841 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:10.941811 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/config-reloader/0.log" Jul 08 17:03:10.970557 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:10.970528 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/kube-rbac-proxy-web/0.log" Jul 08 17:03:11.000729 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.000704 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/kube-rbac-proxy/0.log" Jul 08 17:03:11.027776 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.027747 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/kube-rbac-proxy-metric/0.log" Jul 08 17:03:11.080601 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.080576 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/prom-label-proxy/0.log" Jul 08 17:03:11.112835 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.112811 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c8e8d78-7829-499d-ad86-cc79d27e2d40/init-config-reloader/0.log" Jul 08 17:03:11.434140 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.434114 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-smmh6_3dbf3d66-3020-4891-af4b-06f1b4b072d8/node-exporter/0.log" Jul 08 17:03:11.458846 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.458823 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-smmh6_3dbf3d66-3020-4891-af4b-06f1b4b072d8/kube-rbac-proxy/0.log" Jul 08 17:03:11.479763 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.479736 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-smmh6_3dbf3d66-3020-4891-af4b-06f1b4b072d8/init-textfile/0.log" Jul 08 17:03:11.692720 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.692642 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/prometheus/0.log" Jul 08 17:03:11.728499 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.728472 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/config-reloader/0.log" Jul 08 17:03:11.786357 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.786326 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/thanos-sidecar/0.log" Jul 08 17:03:11.822285 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.822255 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/kube-rbac-proxy-web/0.log" Jul 08 17:03:11.862775 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.862747 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/kube-rbac-proxy/0.log" Jul 08 17:03:11.895025 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.894995 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/kube-rbac-proxy-thanos/0.log" Jul 08 17:03:11.925017 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:11.924982 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_71ce3ab8-2941-4e75-98e3-9d9d6c4aa89c/init-config-reloader/0.log" Jul 08 17:03:12.061842 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:12.061818 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-7vx6w_6c17e049-6ed0-4703-b8ec-b68606678591/prometheus-operator-admission-webhook/0.log" Jul 08 17:03:12.108913 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:12.108890 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78d56bcd6c-qlhrf_5cd01afb-1f34-4bc0-b447-715f5df04076/telemeter-client/0.log" Jul 08 17:03:12.142678 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:12.142653 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78d56bcd6c-qlhrf_5cd01afb-1f34-4bc0-b447-715f5df04076/reload/0.log" Jul 08 17:03:12.165912 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:12.165883 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78d56bcd6c-qlhrf_5cd01afb-1f34-4bc0-b447-715f5df04076/kube-rbac-proxy/0.log" Jul 08 17:03:13.905863 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.905827 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc"] Jul 08 17:03:13.906355 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.906287 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="gather" Jul 08 17:03:13.906355 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.906305 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="gather" Jul 08 17:03:13.906355 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.906318 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="copy" Jul 08 17:03:13.906355 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.906325 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="copy" Jul 08 17:03:13.906554 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.906399 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="gather" Jul 08 17:03:13.906554 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.906413 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7637fcd-c2d9-4075-825b-327a811fe8e1" containerName="copy" Jul 08 17:03:13.910661 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.910639 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:13.913122 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.913099 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gsbb8\"/\"kube-root-ca.crt\"" Jul 08 17:03:13.913946 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.913924 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gsbb8\"/\"default-dockercfg-k8kqc\"" Jul 08 17:03:13.914016 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.913924 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gsbb8\"/\"openshift-service-ca.crt\"" Jul 08 17:03:13.921773 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:13.921747 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc"] Jul 08 17:03:14.106984 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.106955 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-sys\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.107133 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.107012 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-proc\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.107133 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.107064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jgjz\" (UniqueName: \"kubernetes.io/projected/59e6c905-4ba5-4ad0-9521-26eafd4308a4-kube-api-access-5jgjz\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.107238 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.107191 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-lib-modules\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.107238 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.107212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-podres\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208126 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-lib-modules\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208126 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208078 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-podres\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208126 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-sys\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208389 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-proc\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208389 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jgjz\" (UniqueName: \"kubernetes.io/projected/59e6c905-4ba5-4ad0-9521-26eafd4308a4-kube-api-access-5jgjz\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208389 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208239 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-podres\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208389 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208239 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-sys\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208389 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208260 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-proc\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.208389 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.208262 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59e6c905-4ba5-4ad0-9521-26eafd4308a4-lib-modules\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.216608 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.216587 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jgjz\" (UniqueName: \"kubernetes.io/projected/59e6c905-4ba5-4ad0-9521-26eafd4308a4-kube-api-access-5jgjz\") pod \"perf-node-gather-daemonset-jsjfc\" (UID: \"59e6c905-4ba5-4ad0-9521-26eafd4308a4\") " pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.219514 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.219486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.378824 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.378791 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-zjlt6_3090f41d-918d-42ac-b7ae-837a2cb25685/download-server/0.log" Jul 08 17:03:14.387425 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.387406 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc"] Jul 08 17:03:14.389968 ip-10-0-143-156 kubenswrapper[2563]: W0708 17:03:14.389942 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod59e6c905_4ba5_4ad0_9521_26eafd4308a4.slice/crio-b920779b51780d1c640a466f25541f46f8646315f2b00b5a7c72563e67519d50 WatchSource:0}: Error finding container b920779b51780d1c640a466f25541f46f8646315f2b00b5a7c72563e67519d50: Status 404 returned error can't find the container with id b920779b51780d1c640a466f25541f46f8646315f2b00b5a7c72563e67519d50 Jul 08 17:03:14.503357 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.503268 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" event={"ID":"59e6c905-4ba5-4ad0-9521-26eafd4308a4","Type":"ContainerStarted","Data":"7bf1a5ce456d0356998f7a1643f1e8f2760abad99cd66324b61305241225f5ab"} Jul 08 17:03:14.503357 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.503327 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" event={"ID":"59e6c905-4ba5-4ad0-9521-26eafd4308a4","Type":"ContainerStarted","Data":"b920779b51780d1c640a466f25541f46f8646315f2b00b5a7c72563e67519d50"} Jul 08 17:03:14.503547 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.503406 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:14.543790 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:14.543735 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" podStartSLOduration=1.543718305 podStartE2EDuration="1.543718305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:03:14.542842777 +0000 UTC m=+406.305834636" watchObservedRunningTime="2026-07-08 17:03:14.543718305 +0000 UTC m=+406.306710163" Jul 08 17:03:15.770876 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:15.770845 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x7crb_3e40726d-a930-4435-9ba2-d7ede73e48a8/dns/0.log" Jul 08 17:03:15.797254 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:15.797227 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x7crb_3e40726d-a930-4435-9ba2-d7ede73e48a8/kube-rbac-proxy/0.log" Jul 08 17:03:15.845472 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:15.845432 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bc62n_6007f44b-deb5-47d6-a9ba-53ea03fe32e6/dns-node-resolver/0.log" Jul 08 17:03:16.301840 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:16.301799 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7985fdcb94-hbsms_0588eff6-ffcc-4062-9bad-cc308888dc9f/registry/0.log" Jul 08 17:03:16.324552 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:16.324515 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7tb8p_c752ba3a-30e7-4d5a-85f8-eea123b359aa/node-ca/0.log" Jul 08 17:03:17.525408 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:17.525380 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brkmd_dd0ce6a4-e00e-4f12-aba0-5524ccbac037/serve-healthcheck-canary/0.log" Jul 08 17:03:18.112885 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:18.112855 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqtkk_8fecbdd8-5abd-4b47-9e87-2957eda23b79/kube-rbac-proxy/0.log" Jul 08 17:03:18.138508 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:18.138483 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqtkk_8fecbdd8-5abd-4b47-9e87-2957eda23b79/exporter/0.log" Jul 08 17:03:18.164565 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:18.164531 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqtkk_8fecbdd8-5abd-4b47-9e87-2957eda23b79/extractor/0.log" Jul 08 17:03:20.516701 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:20.516676 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gsbb8/perf-node-gather-daemonset-jsjfc" Jul 08 17:03:20.930849 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:20.930763 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:21.013049 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:21.013018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:21.087173 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:21.087112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:21.666050 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:21.666018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:21.745627 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:21.745602 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bb8gf_1c562f70-0d3f-4bb9-8092-56fcdefbdb3c/konnectivity-agent/0.log" limit=1024 Jul 08 17:03:23.571650 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:23.571622 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vgwjr_03e9c757-1324-4ebc-be00-bce7536d7986/migrator/0.log" Jul 08 17:03:23.648384 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:23.648355 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vgwjr_03e9c757-1324-4ebc-be00-bce7536d7986/graceful-termination/0.log" Jul 08 17:03:24.934326 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:24.934294 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5bxrg_6aaa8e6e-cad3-4b39-af54-ea56b705ca77/kube-multus/0.log" Jul 08 17:03:24.993466 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:24.993435 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/kube-multus-additional-cni-plugins/0.log" Jul 08 17:03:25.018248 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.018163 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/egress-router-binary-copy/0.log" Jul 08 17:03:25.050141 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.050118 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/cni-plugins/0.log" Jul 08 17:03:25.075287 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.075265 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/bond-cni-plugin/0.log" Jul 08 17:03:25.102363 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.102334 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/routeoverride-cni/0.log" Jul 08 17:03:25.128640 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.128612 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/whereabouts-cni-bincopy/0.log" Jul 08 17:03:25.154694 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.154651 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qkvzz_f6504eba-a3f1-416f-b1df-56308e89d459/whereabouts-cni/0.log" Jul 08 17:03:25.735050 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.735021 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nkwpj_747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687/network-metrics-daemon/0.log" Jul 08 17:03:25.758222 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:25.758187 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nkwpj_747c5ae0-7f2c-4cfd-81e1-c8dbecf1f687/kube-rbac-proxy/0.log" Jul 08 17:03:26.758400 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.758366 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/ovn-controller/0.log" Jul 08 17:03:26.778801 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.778774 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/ovn-acl-logging/0.log" Jul 08 17:03:26.806277 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.806243 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/kube-rbac-proxy-node/0.log" Jul 08 17:03:26.835864 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.835836 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 17:03:26.858253 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.858222 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/northd/0.log" Jul 08 17:03:26.880877 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.880848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/nbdb/0.log" Jul 08 17:03:26.908272 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.908244 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/sbdb/0.log" Jul 08 17:03:26.978316 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:26.978284 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-px95b_c24715a0-4a60-4813-b316-57aaa69e6247/ovnkube-controller/0.log" Jul 08 17:03:28.267413 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:28.267379 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-pb29q_12a12eb7-e5a5-42f0-9a59-0c70f6e87268/network-check-target-container/0.log" Jul 08 17:03:29.224037 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:29.224009 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-f78g2_efd8aa5d-ccf1-4b87-9e54-a256d619c7e8/iptables-alerter/0.log" Jul 08 17:03:29.934949 ip-10-0-143-156 kubenswrapper[2563]: I0708 17:03:29.934875 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4dtj9_0922e014-26df-43c7-93b0-dc3267e0c109/tuned/0.log"