Jul 21 01:51:46.080383 ip-10-0-135-168 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 01:51:46.080470 ip-10-0-135-168 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 01:51:46.080479 ip-10-0-135-168 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 01:51:46.080765 ip-10-0-135-168 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 01:51:56.292444 ip-10-0-135-168 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 01:51:56.292459 ip-10-0-135-168 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b10ca9a6d74e49e89a5605b78f8da266 -- Jul 21 01:53:32.050660 ip-10-0-135-168 systemd[1]: Starting Kubernetes Kubelet... Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.506713 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512131 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512144 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512147 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512150 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512153 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512156 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512159 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 01:53:32.540811 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512162 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512168 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512173 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512176 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512179 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512183 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512186 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512189 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512192 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512195 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512198 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512200 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512203 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512206 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512208 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512211 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512213 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512216 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512219 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512222 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 01:53:32.542975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512224 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512227 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512229 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512232 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512235 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512237 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512239 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512242 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512245 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512247 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512250 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512252 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512254 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512257 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512260 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512262 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512264 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512267 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512271 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512273 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 01:53:32.545237 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512277 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512280 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512282 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512285 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512288 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512290 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512293 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512303 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512305 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512308 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512311 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512314 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512316 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512319 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512321 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512324 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512327 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512330 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512332 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 01:53:32.559019 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512335 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512338 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512340 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512343 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512345 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512348 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512350 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512353 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512356 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512360 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512364 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512367 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512370 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512374 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512377 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512380 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512382 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512387 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512389 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 01:53:32.561180 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512392 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512395 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512400 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512420 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512423 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512426 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512429 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512432 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512434 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512438 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512440 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512443 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512446 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.512448 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515259 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515270 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515274 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515278 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515280 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515283 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 01:53:32.562935 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515286 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515288 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515291 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515295 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515300 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515302 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515305 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515308 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515310 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515313 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515316 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515319 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515321 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515324 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515327 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515330 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515333 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515335 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515338 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 01:53:32.565361 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515340 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515348 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515351 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515353 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515356 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515358 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515361 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515366 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515369 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515372 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515375 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515378 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515382 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515385 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515388 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515390 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515393 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515395 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515398 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515400 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 01:53:32.579035 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515422 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515425 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515428 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515430 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515433 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515436 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515439 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515442 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515445 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515447 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515450 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515453 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515456 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515458 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515461 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515465 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515469 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515472 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515474 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515477 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 01:53:32.581463 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515480 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 01:53:32.582217 ip-10-0-135-168 systemd[1]: Started Kubernetes Kubelet. Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515483 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515485 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515489 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515492 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515494 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515497 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515500 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515503 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515505 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515508 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515511 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515513 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515516 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515518 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515521 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515525 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515527 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515530 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 01:53:32.584733 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515533 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515535 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515538 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515541 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515543 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515546 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515549 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515551 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515555 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515557 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515560 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515562 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515565 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515567 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515570 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515947 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515952 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515955 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515957 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 01:53:32.599470 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515960 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515963 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515966 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515968 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515971 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515974 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515977 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515980 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515982 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515985 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515988 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515991 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515994 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515997 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.515999 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516002 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516004 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516006 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516010 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516013 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 01:53:32.603178 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516016 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516018 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516021 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516023 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516026 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516031 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516033 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516036 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516040 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516043 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516045 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516048 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516051 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516054 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516056 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516058 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516061 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516064 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516066 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 01:53:32.605834 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516069 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516071 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516074 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516076 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516079 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516082 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516085 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516088 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516091 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516093 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516096 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516098 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516101 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516106 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516109 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516112 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516115 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516118 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516121 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516124 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 01:53:32.607997 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516126 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516129 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516131 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516134 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516137 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516140 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516143 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516145 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516148 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516150 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516153 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516155 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516158 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516160 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516163 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516166 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516168 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516171 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516173 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516176 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 01:53:32.610423 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516179 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516182 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516184 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516187 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516190 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516192 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516196 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516198 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516200 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516203 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516207 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516209 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516212 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516214 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516218 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516222 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516304 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516314 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516324 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516331 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516339 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 01:53:32.622972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516346 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516355 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516359 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516363 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516366 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516370 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516373 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516376 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516378 2561 flags.go:64] FLAG: --cgroup-root="" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516381 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516384 2561 flags.go:64] FLAG: --client-ca-file="" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516388 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516391 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516395 2561 flags.go:64] FLAG: --cluster-domain="" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516398 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516416 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516420 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516423 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516427 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516429 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516433 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516437 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516440 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516443 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 01:53:32.625586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516446 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516449 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516452 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516456 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516459 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516462 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516465 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516468 2561 flags.go:64] FLAG: --enable-server="true" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516472 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516477 2561 flags.go:64] FLAG: --event-burst="100" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516480 2561 flags.go:64] FLAG: --event-qps="50" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516482 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516485 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516488 2561 flags.go:64] FLAG: --eviction-hard="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516492 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516495 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516498 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516501 2561 flags.go:64] FLAG: --eviction-soft="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516504 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516506 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516510 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516513 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516516 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516519 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516521 2561 flags.go:64] FLAG: --feature-gates="" Jul 21 01:53:32.639309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516525 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516528 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516531 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516534 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516538 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516541 2561 flags.go:64] FLAG: --help="false" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516544 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-135-168.ec2.internal" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516547 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516550 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516553 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516556 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516559 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516563 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516566 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516568 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516573 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516575 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516579 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516582 2561 flags.go:64] FLAG: --kube-reserved="" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516584 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516587 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516594 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516597 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516600 2561 flags.go:64] FLAG: --lock-file="" Jul 21 01:53:32.642334 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516602 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516605 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516608 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516614 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516617 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516620 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516623 2561 flags.go:64] FLAG: --logging-format="text" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516626 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516629 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516632 2561 flags.go:64] FLAG: --manifest-url="" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516634 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516639 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516643 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516647 2561 flags.go:64] FLAG: --max-pods="110" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516650 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516653 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516656 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516659 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516662 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516665 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516668 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516675 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516678 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516682 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 01:53:32.884421 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516685 2561 flags.go:64] FLAG: --pod-cidr="" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516688 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516693 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516696 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516699 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516701 2561 flags.go:64] FLAG: --port="10250" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516704 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516707 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0356827fa2e6f4a73" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516711 2561 flags.go:64] FLAG: --qos-reserved="" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516714 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516717 2561 flags.go:64] FLAG: --register-node="true" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516720 2561 flags.go:64] FLAG: --register-with-taints="" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516723 2561 flags.go:64] FLAG: --registry-burst="10" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516727 2561 flags.go:64] FLAG: --registry-qps="5" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516730 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516732 2561 flags.go:64] FLAG: --reserved-memory="" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516736 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516739 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516742 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516745 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516747 2561 flags.go:64] FLAG: --runonce="false" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516752 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516755 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516758 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516761 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 01:53:32.885822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516763 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516766 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516769 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516773 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516776 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516778 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516784 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516787 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516790 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516792 2561 flags.go:64] FLAG: --system-cgroups="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516795 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516800 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516803 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516806 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516810 2561 flags.go:64] FLAG: --tls-min-version="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516812 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516815 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516818 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516820 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516823 2561 flags.go:64] FLAG: --v="2" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516827 2561 flags.go:64] FLAG: --version="false" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516831 2561 flags.go:64] FLAG: --vmodule="" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516835 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.516838 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516954 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 01:53:32.892955 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516959 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516963 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516966 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516970 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516973 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516975 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516978 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516981 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516984 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516987 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516989 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516992 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.516994 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517000 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517002 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517005 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517007 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517010 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517012 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517015 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 01:53:32.893792 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517018 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517021 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517023 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517026 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517028 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517031 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517033 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517036 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517039 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517042 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517045 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517047 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517050 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517052 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517055 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517059 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517062 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517065 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517067 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517070 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 01:53:32.894453 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517072 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517075 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517077 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517080 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517082 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517087 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517089 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517092 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517094 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517097 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517100 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517102 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517104 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517107 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517109 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517112 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517115 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517117 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517120 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517122 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 01:53:32.895093 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517125 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517127 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517130 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517132 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517135 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517137 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517140 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517144 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517146 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517149 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517151 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517154 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517157 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517159 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517161 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517164 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517167 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517172 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517175 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 01:53:32.895721 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517178 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517182 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517185 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517187 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517190 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517193 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517197 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517201 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517205 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517208 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517211 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517214 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517217 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517220 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517222 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517225 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517228 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517230 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 01:53:32.896275 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.517233 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.518110 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.525135 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.525155 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.525175 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.525181 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525238 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525243 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525246 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525249 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525252 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525255 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525258 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525261 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525264 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 01:53:32.896773 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525267 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525270 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525275 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525278 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525280 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525283 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525286 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525289 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525292 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525295 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525297 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525300 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525303 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525305 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525308 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525310 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525313 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525316 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525318 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525321 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 01:53:32.897256 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525324 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525327 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525330 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525332 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525335 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525338 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525341 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525346 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525349 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525352 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525355 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525357 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525360 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525362 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525365 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525368 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525371 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525374 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525376 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525379 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 01:53:32.897803 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525382 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525384 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525388 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525392 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525396 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525400 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525418 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525422 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525425 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525428 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525431 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525434 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525436 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525440 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525443 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525446 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525448 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525451 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525453 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 01:53:32.898377 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525456 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525459 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525461 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525464 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525467 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525470 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525472 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525478 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525481 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525483 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525486 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525489 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525491 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525494 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525496 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525499 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525502 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525505 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525507 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 01:53:32.898901 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525511 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525515 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525518 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525520 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525523 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525526 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525528 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525531 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525534 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525537 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525539 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525542 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.525550 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525664 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525668 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 01:53:32.899460 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525670 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525673 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525678 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525680 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525683 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525687 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525690 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525692 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525695 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525699 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525703 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525706 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525709 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525712 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525715 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525717 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525720 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525722 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525725 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 01:53:32.899875 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525727 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525730 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525733 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525736 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525738 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525741 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525744 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525746 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525749 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525752 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525756 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525759 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525762 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525765 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525768 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525772 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525774 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525777 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525781 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525783 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 01:53:32.900435 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525786 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525788 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525791 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525793 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525796 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525799 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525801 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525804 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525807 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525809 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525812 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525816 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525820 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525823 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525825 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525828 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525831 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525833 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525836 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525838 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 01:53:32.900975 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525841 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525844 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525846 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525849 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525852 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525854 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525857 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525861 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525864 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525868 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525873 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525876 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525879 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525882 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525885 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525888 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525890 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525893 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525896 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 01:53:32.901566 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525898 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525901 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525903 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525917 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525920 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525923 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525926 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525928 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525931 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525934 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525936 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525939 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525942 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525944 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525947 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525950 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525953 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525955 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 01:53:32.902192 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:32.525958 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.525963 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.526069 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.528309 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.529882 2561 server.go:1019] "Starting client certificate rotation" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.529985 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.530828 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.533228 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.535206 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.546973 2561 log.go:25] "Validated CRI v1 runtime API" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.549677 2561 log.go:25] "Validated CRI v1 image API" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.551357 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 01:53:32.903090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.554018 2561 fs.go:135] Filesystem UUIDs: map[68fd765b-66c4-4efa-a83d-0a744381d524:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cd3cb7b7-8d19-454e-affe-90a9c2c3ec2a:/dev/nvme0n1p3] Jul 21 01:53:32.903701 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.554033 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.559829 2561 manager.go:216] Machine: {Timestamp:2026-07-21 01:53:32.558455782 +0000 UTC m=+0.375490433 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097499 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec216b9b5bf33dd5c2c39a468550c046 SystemUUID:ec216b9b-5bf3-3dd5-c2c3-9a468550c046 BootID:b10ca9a6-d74e-49e8-9a56-05b78f8da266 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e3:63:01:7a:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e3:63:01:7a:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:3f:a4:d2:58:f4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.559923 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.560024 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.561288 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.563710 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.563737 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 01:53:32.903905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.564626 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.564636 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.564644 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.564658 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.566473 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.569784 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.569903 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.569943 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.569964 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.569987 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.570000 2561 kubelet.go:398] "Adding apiserver pod source" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.570019 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.570401 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.570436 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.572944 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.575496 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.575523 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577544 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577563 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577570 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577575 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577614 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577625 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577635 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577640 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577649 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577655 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577664 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577669 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577693 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.577698 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.579107 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.579177 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.579249 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.581439 2561 server.go:1268] "Started kubelet" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.581531 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.581615 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.581677 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.583168 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 21 01:53:33.038014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.584160 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.585956 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.586069 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-168.ec2.internal.18c42b2a327f8c8d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-168.ec2.internal,UID:ip-10-0-135-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-168.ec2.internal,},FirstTimestamp:2026-07-21 01:53:32.581399693 +0000 UTC m=+0.398434323,LastTimestamp:2026-07-21 01:53:32.581399693 +0000 UTC m=+0.398434323,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-168.ec2.internal,}" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.593384 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.593394 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.593459 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.594331 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.594309 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.594428 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.594570 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.594578 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595807 2561 factory.go:153] Registering CRI-O factory Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595823 2561 factory.go:223] Registration of the crio container factory successfully Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595872 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595881 2561 factory.go:55] Registering systemd factory Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595890 2561 factory.go:223] Registration of the systemd container factory successfully Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595909 2561 factory.go:103] Registering Raw factory Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.595929 2561 manager.go:1196] Started watching for new ooms in manager Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.596364 2561 manager.go:349] Starting recovery of all containers Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.607291 2561 manager.go:354] Recovery completed Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.607399 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9fd9k" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.607976 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.608020 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.611982 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.614612 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.614642 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.614656 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.615106 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.615114 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.615127 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.615166 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.615379 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9fd9k" Jul 21 01:53:33.039814 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.616306 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.616338 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.616353 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.617391 2561 policy_none.go:49] "None policy: Start" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.617423 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.617435 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.618335 2561 policy_none.go:47] "Start" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.668926 2561 manager.go:337] "Starting Device Plugin manager" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.668976 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.668985 2561 server.go:87] "Starting device plugin registration server" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.669210 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.669223 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.669313 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.669386 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.669393 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.669860 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.669901 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.700642 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.701792 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.701825 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.701842 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.701870 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.704062 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.770180 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.771025 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.771049 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.771064 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.771090 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.782618 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.782635 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-168.ec2.internal\": node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.802345 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal"] Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.802420 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.802496 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.806274 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.806295 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.806310 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.810857 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.810997 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.047346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811028 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811580 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811604 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811616 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811647 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811665 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.811675 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.818121 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.818145 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.818818 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientMemory" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.818842 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.818856 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeHasSufficientPID" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.839935 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-168.ec2.internal\" not found" node="ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.844123 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-168.ec2.internal\" not found" node="ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.896316 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3c5fff1526cc232660f7475ed460e6b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal\" (UID: \"d3c5fff1526cc232660f7475ed460e6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.896337 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3c5fff1526cc232660f7475ed460e6b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal\" (UID: \"d3c5fff1526cc232660f7475ed460e6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.896355 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf74ce26f0de05ee8701a049d7ed8096-config\") pod \"kube-apiserver-proxy-ip-10-0-135-168.ec2.internal\" (UID: \"cf74ce26f0de05ee8701a049d7ed8096\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:32.903415 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.996704 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf74ce26f0de05ee8701a049d7ed8096-config\") pod \"kube-apiserver-proxy-ip-10-0-135-168.ec2.internal\" (UID: \"cf74ce26f0de05ee8701a049d7ed8096\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.996734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3c5fff1526cc232660f7475ed460e6b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal\" (UID: \"d3c5fff1526cc232660f7475ed460e6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.996750 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3c5fff1526cc232660f7475ed460e6b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal\" (UID: \"d3c5fff1526cc232660f7475ed460e6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.996783 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3c5fff1526cc232660f7475ed460e6b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal\" (UID: \"d3c5fff1526cc232660f7475ed460e6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049950 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.996808 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cf74ce26f0de05ee8701a049d7ed8096-config\") pod \"kube-apiserver-proxy-ip-10-0-135-168.ec2.internal\" (UID: \"cf74ce26f0de05ee8701a049d7ed8096\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049950 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:32.996808 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3c5fff1526cc232660f7475ed460e6b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal\" (UID: \"d3c5fff1526cc232660f7475ed460e6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.049950 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.003811 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.147345 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.107324 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.147345 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.143472 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.147345 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.146152 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.231695 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.214002 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.359194 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.314793 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.498561 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.415335 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.516692 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.528915 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.529071 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:33.616736 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-168.ec2.internal\" not found" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.621210 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 01:48:33 +0000 UTC" deadline="2028-03-11 10:01:15.546595819 +0000 UTC" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.621241 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14384h7m41.925358786s" Jul 21 01:53:33.686795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.672977 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 01:53:33.694964 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.694002 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.812000 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.709519 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 01:53:33.812000 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.711453 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" Jul 21 01:53:33.812000 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.720248 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 01:53:33.953921 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.890866 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 01:53:33.953921 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:33.905395 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf74ce26f0de05ee8701a049d7ed8096.slice/crio-a7a32d70700ae4304b52d7d550f7c21c797d9babc841a4940097494ebaa9c37a WatchSource:0}: Error finding container a7a32d70700ae4304b52d7d550f7c21c797d9babc841a4940097494ebaa9c37a: Status 404 returned error can't find the container with id a7a32d70700ae4304b52d7d550f7c21c797d9babc841a4940097494ebaa9c37a Jul 21 01:53:33.953921 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:33.906042 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c5fff1526cc232660f7475ed460e6b.slice/crio-1c792b018bd250e7081ab819c2a1af67b1980c993932513deb12f9a27268431c WatchSource:0}: Error finding container 1c792b018bd250e7081ab819c2a1af67b1980c993932513deb12f9a27268431c: Status 404 returned error can't find the container with id 1c792b018bd250e7081ab819c2a1af67b1980c993932513deb12f9a27268431c Jul 21 01:53:33.953921 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.910840 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 01:53:33.953921 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:33.913789 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 01:53:34.572000 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.571972 2561 apiserver.go:52] "Watching apiserver" Jul 21 01:53:34.577220 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.577197 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 01:53:34.577650 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.577618 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j","openshift-dns/node-resolver-lf6nt","openshift-image-registry/node-ca-d49sq","openshift-network-diagnostics/network-check-target-m64bz","kube-system/global-pull-secret-syncer-wt28r","openshift-cluster-node-tuning-operator/tuned-2mwl5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal","openshift-multus/multus-9klcj","openshift-multus/multus-additional-cni-plugins-d9tm7","openshift-multus/network-metrics-daemon-d6brs","openshift-network-operator/iptables-alerter-9289p","openshift-ovn-kubernetes/ovnkube-node-qfcwj","kube-system/konnectivity-agent-2pmdj","kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal"] Jul 21 01:53:34.579222 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.579204 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:34.579325 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.579298 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:34.580380 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.580359 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.581981 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.581962 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.582772 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.582748 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 01:53:34.582868 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.582779 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 01:53:34.582868 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.582762 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-82qt6\"" Jul 21 01:53:34.583360 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.583340 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:34.583475 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.583411 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:34.584182 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.584063 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 01:53:34.584182 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.584105 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lmgwf\"" Jul 21 01:53:34.584471 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.584452 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.584548 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.584523 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.586839 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.585582 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.586839 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.585707 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:34.586956 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.586912 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.588468 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.588444 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.589090 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.589067 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j8vwl\"" Jul 21 01:53:34.589327 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.589309 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.589327 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.589339 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.589664 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.589640 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.590380 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.590238 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.590380 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.590245 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 01:53:34.591065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.590733 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 01:53:34.591065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.590791 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.591065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.590831 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lp26h\"" Jul 21 01:53:34.591065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.590958 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 01:53:34.591395 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.591151 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.591977 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.591769 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 01:53:34.591977 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.591802 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rls7d\"" Jul 21 01:53:34.592553 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.592533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.593119 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.593100 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m7ccq\"" Jul 21 01:53:34.593205 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.593136 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.593205 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.593136 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.594235 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.594146 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.594235 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.594185 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 01:53:34.594235 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.594218 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.594491 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.594469 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m6w29\"" Jul 21 01:53:34.594491 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.594487 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 01:53:34.594794 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.594764 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.595641 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.595621 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.596329 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.596128 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fzgwr\"" Jul 21 01:53:34.596329 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.596246 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.596329 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.596330 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 01:53:34.596988 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.596968 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.597710 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.597689 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 01:53:34.597918 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.597899 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 01:53:34.598198 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.598174 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 01:53:34.598309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.598182 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 01:53:34.598375 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.598214 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 01:53:34.598441 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.598217 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z4w66\"" Jul 21 01:53:34.598515 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.598231 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 01:53:34.605443 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.605423 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 01:53:34.626048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.625998 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-cni-binary-copy\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.626152 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-cni-binary-copy\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.626222 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.626222 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626203 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-kubelet-config\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.626310 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-run\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.626829 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovn-node-metrics-cert\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.626929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-registration-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.626929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-device-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.626929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:34.627106 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626938 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.627106 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626959 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysctl-d\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.627106 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.626981 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-627nf\" (UniqueName: \"kubernetes.io/projected/9dbab70a-e2fa-4489-b191-4eed783b55a9-kube-api-access-627nf\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.627106 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627080 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-modprobe-d\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.627106 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627103 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-var-lib-kubelet\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-system-cni-dir\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-netns\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627184 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-sys-fs\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96fgk\" (UniqueName: \"kubernetes.io/projected/e80b8395-410e-409c-8e77-7f671ccb99d7-kube-api-access-96fgk\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627255 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysconfig\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627292 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-systemd\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627317 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-cnibin\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627340 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-log-socket\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.627363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627365 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-cni-bin\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627389 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-cni-netd\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovnkube-config\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627453 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-hostroot\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-etc-kubernetes\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-dbus\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627577 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-sys\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627602 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc2cj\" (UniqueName: \"kubernetes.io/projected/0f019e58-fba6-424e-8878-3aba3b3f1e01-kube-api-access-wc2cj\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627637 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovnkube-script-lib\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7fpk\" (UniqueName: \"kubernetes.io/projected/956c97dd-8b8b-4dd0-b22b-43110503e7f7-kube-api-access-k7fpk\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627697 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-cnibin\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627749 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-os-release\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.627838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627774 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/883b0920-9cd4-4e33-9fa0-2de1dfb79720-tmp\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627821 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-serviceca\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-system-cni-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-cni-multus\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627895 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-kubelet\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-multus-certs\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627952 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq897\" (UniqueName: \"kubernetes.io/projected/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-kube-api-access-bq897\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.627984 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-os-release\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-systemd\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628044 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-env-overrides\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-k8s-cni-cncf-io\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628080 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-cni-bin\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628095 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d36da6-d97f-400d-bb0d-2d8986690d0c-tmp-dir\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.628466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysctl-conf\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628124 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-lib-modules\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628140 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-tuned\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-node-log\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bde4cbaf-9131-4510-81a3-5d9daf8f801a-agent-certs\") pod \"konnectivity-agent-2pmdj\" (UID: \"bde4cbaf-9131-4510-81a3-5d9daf8f801a\") " pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628210 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67d36da6-d97f-400d-bb0d-2d8986690d0c-hosts-file\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628226 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f019e58-fba6-424e-8878-3aba3b3f1e01-iptables-alerter-script\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-kubelet\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628257 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-etc-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628279 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k29jm\" (UniqueName: \"kubernetes.io/projected/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-kube-api-access-k29jm\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628296 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-host\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628317 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qmqd\" (UniqueName: \"kubernetes.io/projected/e732d16d-8b81-453f-b166-bf540e40195a-kube-api-access-7qmqd\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628368 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-daemon-config\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.629048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628393 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-kubernetes\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-systemd-units\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-slash\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628465 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-run-netns\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628513 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-var-lib-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628548 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-ovn\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628574 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628601 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-socket-dir-parent\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628676 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmhw\" (UniqueName: \"kubernetes.io/projected/883b0920-9cd4-4e33-9fa0-2de1dfb79720-kube-api-access-wnmhw\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628692 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628712 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f019e58-fba6-424e-8878-3aba3b3f1e01-host-slash\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.629744 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bde4cbaf-9131-4510-81a3-5d9daf8f801a-konnectivity-ca\") pod \"konnectivity-agent-2pmdj\" (UID: \"bde4cbaf-9131-4510-81a3-5d9daf8f801a\") " pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.630512 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628761 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-host\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.630512 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-cni-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.630512 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-conf-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.630512 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.628844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8hn\" (UniqueName: \"kubernetes.io/projected/67d36da6-d97f-400d-bb0d-2d8986690d0c-kube-api-access-xq8hn\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.630512 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.629006 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-socket-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.647537 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.647505 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpbs9" Jul 21 01:53:34.656474 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.656453 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpbs9" Jul 21 01:53:34.695867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.695771 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 01:53:34.707361 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.707276 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" event={"ID":"cf74ce26f0de05ee8701a049d7ed8096","Type":"ContainerStarted","Data":"a7a32d70700ae4304b52d7d550f7c21c797d9babc841a4940097494ebaa9c37a"} Jul 21 01:53:34.710025 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.709970 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" event={"ID":"d3c5fff1526cc232660f7475ed460e6b","Type":"ContainerStarted","Data":"1c792b018bd250e7081ab819c2a1af67b1980c993932513deb12f9a27268431c"} Jul 21 01:53:34.729731 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729704 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-kubernetes\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.729836 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-systemd-units\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.729899 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-slash\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.729954 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-run-netns\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.729954 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729939 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-var-lib-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730063 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-ovn\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730063 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.729991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730063 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.730209 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730107 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730209 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730315 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730212 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-socket-dir-parent\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.730463 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730420 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-socket-dir-parent\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.730569 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730498 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-kubernetes\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.730569 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730540 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-systemd-units\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730677 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730576 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-slash\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730677 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730609 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-run-netns\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730677 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730644 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-var-lib-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730682 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-ovn\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.730838 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.730716 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.731289 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.731266 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmhw\" (UniqueName: \"kubernetes.io/projected/883b0920-9cd4-4e33-9fa0-2de1dfb79720-kube-api-access-wnmhw\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.731371 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.731334 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.731371 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.731331 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.731544 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.731527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f019e58-fba6-424e-8878-3aba3b3f1e01-host-slash\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.731614 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.731560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bde4cbaf-9131-4510-81a3-5d9daf8f801a-konnectivity-ca\") pod \"konnectivity-agent-2pmdj\" (UID: \"bde4cbaf-9131-4510-81a3-5d9daf8f801a\") " pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.732183 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732160 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.732266 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732225 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f019e58-fba6-424e-8878-3aba3b3f1e01-host-slash\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.732326 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732265 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-host\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.732326 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732266 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bde4cbaf-9131-4510-81a3-5d9daf8f801a-konnectivity-ca\") pod \"konnectivity-agent-2pmdj\" (UID: \"bde4cbaf-9131-4510-81a3-5d9daf8f801a\") " pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.732326 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732301 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-cni-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.732559 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732327 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-host\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.732559 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732370 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-conf-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.732559 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732397 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8hn\" (UniqueName: \"kubernetes.io/projected/67d36da6-d97f-400d-bb0d-2d8986690d0c-kube-api-access-xq8hn\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.732559 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-socket-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.732559 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732470 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-cni-binary-copy\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.732559 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-cni-binary-copy\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732567 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-cni-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732607 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-kubelet-config\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-run\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732667 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovn-node-metrics-cert\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-registration-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732695 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-device-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732767 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-registration-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732780 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732807 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732813 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-run\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysctl-d\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.732866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.732859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-627nf\" (UniqueName: \"kubernetes.io/projected/9dbab70a-e2fa-4489-b191-4eed783b55a9-kube-api-access-627nf\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.733598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733107 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-kubelet-config\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.733598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733296 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-cni-binary-copy\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.733598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733367 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-conf-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.733598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-socket-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.733598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733516 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 01:53:34.733598 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.733527 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:34.733913 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.733636 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:53:35.23361615 +0000 UTC m=+3.050650789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:34.733913 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733845 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-device-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.734040 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.733968 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e732d16d-8b81-453f-b166-bf540e40195a-cni-binary-copy\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.734106 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-modprobe-d\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734167 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-var-lib-kubelet\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734167 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734129 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysctl-d\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734167 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-system-cni-dir\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.734322 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734177 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-netns\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.734322 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734213 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-sys-fs\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.734322 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734250 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-modprobe-d\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734322 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96fgk\" (UniqueName: \"kubernetes.io/projected/e80b8395-410e-409c-8e77-7f671ccb99d7-kube-api-access-96fgk\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.734555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734328 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysconfig\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-systemd\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734380 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-cnibin\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.734555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734406 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-log-socket\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.734555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-log-socket\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.734555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734509 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-netns\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734557 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-sys-fs\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734593 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-system-cni-dir\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734625 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-systemd\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734635 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-var-lib-kubelet\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734658 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-cni-bin\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734681 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysconfig\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-cni-netd\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734734 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-cni-bin\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734765 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovnkube-config\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734787 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-cnibin\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734794 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:34.734867 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.734872 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734899 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-hostroot\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734869 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-cni-netd\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.734933 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:35.234915905 +0000 UTC m=+3.051950530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-etc-kubernetes\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734981 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.734977 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-hostroot\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735023 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-dbus\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735040 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735073 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-etc-kubernetes\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-sys\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735140 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc2cj\" (UniqueName: \"kubernetes.io/projected/0f019e58-fba6-424e-8878-3aba3b3f1e01-kube-api-access-wc2cj\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735168 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovnkube-script-lib\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7fpk\" (UniqueName: \"kubernetes.io/projected/956c97dd-8b8b-4dd0-b22b-43110503e7f7-kube-api-access-k7fpk\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:34.735460 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735264 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-cnibin\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-os-release\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/883b0920-9cd4-4e33-9fa0-2de1dfb79720-tmp\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735369 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-serviceca\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735452 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-system-cni-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735502 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-cni-multus\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-kubelet\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735575 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-multus-certs\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq897\" (UniqueName: \"kubernetes.io/projected/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-kube-api-access-bq897\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-os-release\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735658 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-systemd\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735684 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-cnibin\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735731 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-env-overrides\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-k8s-cni-cncf-io\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.736241 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735762 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e732d16d-8b81-453f-b166-bf540e40195a-os-release\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735788 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-cni-bin\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735797 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-os-release\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735833 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d36da6-d97f-400d-bb0d-2d8986690d0c-tmp-dir\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735840 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-systemd\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysctl-conf\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-lib-modules\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-tuned\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-node-log\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.735983 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bde4cbaf-9131-4510-81a3-5d9daf8f801a-agent-certs\") pod \"konnectivity-agent-2pmdj\" (UID: \"bde4cbaf-9131-4510-81a3-5d9daf8f801a\") " pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736120 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d36da6-d97f-400d-bb0d-2d8986690d0c-tmp-dir\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736202 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-k8s-cni-cncf-io\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736251 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-cni-bin\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736306 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-cni-multus\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736417 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-var-lib-kubelet\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736475 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-env-overrides\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736494 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-dbus\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736534 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-sys\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736553 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-host-run-multus-certs\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736614 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e80b8395-410e-409c-8e77-7f671ccb99d7-etc-selinux\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736682 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-sysctl-conf\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67d36da6-d97f-400d-bb0d-2d8986690d0c-hosts-file\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f019e58-fba6-424e-8878-3aba3b3f1e01-iptables-alerter-script\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-kubelet\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736890 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-etc-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k29jm\" (UniqueName: \"kubernetes.io/projected/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-kube-api-access-k29jm\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-node-log\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.736980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-host\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qmqd\" (UniqueName: \"kubernetes.io/projected/e732d16d-8b81-453f-b166-bf540e40195a-kube-api-access-7qmqd\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737055 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.737824 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737086 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-daemon-config\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737186 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-system-cni-dir\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737231 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-etc-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737371 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovnkube-config\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737460 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-host\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737465 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f019e58-fba6-424e-8878-3aba3b3f1e01-iptables-alerter-script\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737498 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/883b0920-9cd4-4e33-9fa0-2de1dfb79720-lib-modules\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737522 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-host-kubelet\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737771 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9dbab70a-e2fa-4489-b191-4eed783b55a9-run-openvswitch\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737810 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovnkube-script-lib\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737822 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/67d36da6-d97f-400d-bb0d-2d8986690d0c-hosts-file\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.737991 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-serviceca\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.738557 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.738261 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-multus-daemon-config\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.741298 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.741273 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:53:34.741404 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.741305 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:53:34.741404 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.741319 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:34.741404 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:34.741396 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:35.241378815 +0000 UTC m=+3.058413452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:34.743251 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.743229 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9dbab70a-e2fa-4489-b191-4eed783b55a9-ovn-node-metrics-cert\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.745611 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.744979 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/883b0920-9cd4-4e33-9fa0-2de1dfb79720-etc-tuned\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.746289 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.746181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmhw\" (UniqueName: \"kubernetes.io/projected/883b0920-9cd4-4e33-9fa0-2de1dfb79720-kube-api-access-wnmhw\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.746565 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.746544 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/883b0920-9cd4-4e33-9fa0-2de1dfb79720-tmp\") pod \"tuned-2mwl5\" (UID: \"883b0920-9cd4-4e33-9fa0-2de1dfb79720\") " pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.746935 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.746888 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bde4cbaf-9131-4510-81a3-5d9daf8f801a-agent-certs\") pod \"konnectivity-agent-2pmdj\" (UID: \"bde4cbaf-9131-4510-81a3-5d9daf8f801a\") " pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.747634 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.747593 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq897\" (UniqueName: \"kubernetes.io/projected/9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a-kube-api-access-bq897\") pod \"multus-9klcj\" (UID: \"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a\") " pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.748597 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.748573 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-627nf\" (UniqueName: \"kubernetes.io/projected/9dbab70a-e2fa-4489-b191-4eed783b55a9-kube-api-access-627nf\") pod \"ovnkube-node-qfcwj\" (UID: \"9dbab70a-e2fa-4489-b191-4eed783b55a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.748749 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.748715 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7fpk\" (UniqueName: \"kubernetes.io/projected/956c97dd-8b8b-4dd0-b22b-43110503e7f7-kube-api-access-k7fpk\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:34.749053 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.749027 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qmqd\" (UniqueName: \"kubernetes.io/projected/e732d16d-8b81-453f-b166-bf540e40195a-kube-api-access-7qmqd\") pod \"multus-additional-cni-plugins-d9tm7\" (UID: \"e732d16d-8b81-453f-b166-bf540e40195a\") " pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.749269 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.749246 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96fgk\" (UniqueName: \"kubernetes.io/projected/e80b8395-410e-409c-8e77-7f671ccb99d7-kube-api-access-96fgk\") pod \"aws-ebs-csi-driver-node-r9k8j\" (UID: \"e80b8395-410e-409c-8e77-7f671ccb99d7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.750988 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.750927 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8hn\" (UniqueName: \"kubernetes.io/projected/67d36da6-d97f-400d-bb0d-2d8986690d0c-kube-api-access-xq8hn\") pod \"node-resolver-lf6nt\" (UID: \"67d36da6-d97f-400d-bb0d-2d8986690d0c\") " pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.750988 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.750937 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc2cj\" (UniqueName: \"kubernetes.io/projected/0f019e58-fba6-424e-8878-3aba3b3f1e01-kube-api-access-wc2cj\") pod \"iptables-alerter-9289p\" (UID: \"0f019e58-fba6-424e-8878-3aba3b3f1e01\") " pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.751828 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.751797 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k29jm\" (UniqueName: \"kubernetes.io/projected/35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c-kube-api-access-k29jm\") pod \"node-ca-d49sq\" (UID: \"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c\") " pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.894742 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.894699 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:34.902950 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.902921 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d49sq" Jul 21 01:53:34.904723 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.904690 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde4cbaf_9131_4510_81a3_5d9daf8f801a.slice/crio-ef00706d869742505e88aaf2763041153abb3cab3d54f758417fdd959077025d WatchSource:0}: Error finding container ef00706d869742505e88aaf2763041153abb3cab3d54f758417fdd959077025d: Status 404 returned error can't find the container with id ef00706d869742505e88aaf2763041153abb3cab3d54f758417fdd959077025d Jul 21 01:53:34.912271 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.912250 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" Jul 21 01:53:34.913553 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.913526 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35bcaf4c_8af4_42cb_8813_ed6fa7f21e7c.slice/crio-35bca43fabd7d8beaca26547470ed12fe85f37f9213c7c75952c86a1319a2674 WatchSource:0}: Error finding container 35bca43fabd7d8beaca26547470ed12fe85f37f9213c7c75952c86a1319a2674: Status 404 returned error can't find the container with id 35bca43fabd7d8beaca26547470ed12fe85f37f9213c7c75952c86a1319a2674 Jul 21 01:53:34.918252 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.918081 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" Jul 21 01:53:34.923296 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.923270 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883b0920_9cd4_4e33_9fa0_2de1dfb79720.slice/crio-176bb2d873ff33afcbbb9d944263202cea23d468bb2456b087b2d33982d5a6c2 WatchSource:0}: Error finding container 176bb2d873ff33afcbbb9d944263202cea23d468bb2456b087b2d33982d5a6c2: Status 404 returned error can't find the container with id 176bb2d873ff33afcbbb9d944263202cea23d468bb2456b087b2d33982d5a6c2 Jul 21 01:53:34.925696 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.925673 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9klcj" Jul 21 01:53:34.928619 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.928586 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode732d16d_8b81_453f_b166_bf540e40195a.slice/crio-35037ed1b7e549464d7a067747f5df0796fed077cb4697d866f490577933881e WatchSource:0}: Error finding container 35037ed1b7e549464d7a067747f5df0796fed077cb4697d866f490577933881e: Status 404 returned error can't find the container with id 35037ed1b7e549464d7a067747f5df0796fed077cb4697d866f490577933881e Jul 21 01:53:34.934272 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.934221 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lf6nt" Jul 21 01:53:34.934904 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.934877 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d67daaf_37d0_4b6a_8f63_fba0e4e51e8a.slice/crio-a450c27494c875bf0816e5934a0f39d0ae6a205059342d648c057725867fab5b WatchSource:0}: Error finding container a450c27494c875bf0816e5934a0f39d0ae6a205059342d648c057725867fab5b: Status 404 returned error can't find the container with id a450c27494c875bf0816e5934a0f39d0ae6a205059342d648c057725867fab5b Jul 21 01:53:34.944040 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.944017 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d36da6_d97f_400d_bb0d_2d8986690d0c.slice/crio-dbfa289c40ea7287143fbb788ccc451dfde565a5eb87e59bfcb166cd70470605 WatchSource:0}: Error finding container dbfa289c40ea7287143fbb788ccc451dfde565a5eb87e59bfcb166cd70470605: Status 404 returned error can't find the container with id dbfa289c40ea7287143fbb788ccc451dfde565a5eb87e59bfcb166cd70470605 Jul 21 01:53:34.945097 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.945075 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" Jul 21 01:53:34.954840 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.954810 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80b8395_410e_409c_8e77_7f671ccb99d7.slice/crio-04c012edd9db3acfa7406df3cf55db48c8334a63e82070e3003f490a8ad20aee WatchSource:0}: Error finding container 04c012edd9db3acfa7406df3cf55db48c8334a63e82070e3003f490a8ad20aee: Status 404 returned error can't find the container with id 04c012edd9db3acfa7406df3cf55db48c8334a63e82070e3003f490a8ad20aee Jul 21 01:53:34.956047 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.955891 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9289p" Jul 21 01:53:34.962765 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:34.962741 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:53:34.965674 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.965650 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f019e58_fba6_424e_8878_3aba3b3f1e01.slice/crio-c26dbe444b3322160815b6dcb00e679804139c0212f2af196be3111c0abfcc73 WatchSource:0}: Error finding container c26dbe444b3322160815b6dcb00e679804139c0212f2af196be3111c0abfcc73: Status 404 returned error can't find the container with id c26dbe444b3322160815b6dcb00e679804139c0212f2af196be3111c0abfcc73 Jul 21 01:53:34.974009 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:53:34.973981 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dbab70a_e2fa_4489_b191_4eed783b55a9.slice/crio-a4bd7d207ce183b870dd3a16062f1cd3bfad71f4722521f827a482e528eb1de0 WatchSource:0}: Error finding container a4bd7d207ce183b870dd3a16062f1cd3bfad71f4722521f827a482e528eb1de0: Status 404 returned error can't find the container with id a4bd7d207ce183b870dd3a16062f1cd3bfad71f4722521f827a482e528eb1de0 Jul 21 01:53:35.058805 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.058778 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 01:53:35.240800 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.240717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:35.240800 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.240761 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:35.241005 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.240877 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:35.241005 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.240947 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:53:36.240932058 +0000 UTC m=+4.057966679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:35.241005 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.240877 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:35.241159 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.241050 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:36.241026439 +0000 UTC m=+4.058061065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:35.341835 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.341799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:35.342007 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.341983 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:53:35.342054 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.342016 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:53:35.342054 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.342028 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:35.342140 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:35.342087 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:36.342067263 +0000 UTC m=+4.159101883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:35.659169 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.659084 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 01:48:34 +0000 UTC" deadline="2028-03-05 13:50:34.468805424 +0000 UTC" Jul 21 01:53:35.659169 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.659115 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14243h56m58.809694389s" Jul 21 01:53:35.713236 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.713199 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"a4bd7d207ce183b870dd3a16062f1cd3bfad71f4722521f827a482e528eb1de0"} Jul 21 01:53:35.714562 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.714532 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerStarted","Data":"35037ed1b7e549464d7a067747f5df0796fed077cb4697d866f490577933881e"} Jul 21 01:53:35.715892 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.715860 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" event={"ID":"883b0920-9cd4-4e33-9fa0-2de1dfb79720","Type":"ContainerStarted","Data":"176bb2d873ff33afcbbb9d944263202cea23d468bb2456b087b2d33982d5a6c2"} Jul 21 01:53:35.717068 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.717041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d49sq" event={"ID":"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c","Type":"ContainerStarted","Data":"35bca43fabd7d8beaca26547470ed12fe85f37f9213c7c75952c86a1319a2674"} Jul 21 01:53:35.718287 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.718261 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2pmdj" event={"ID":"bde4cbaf-9131-4510-81a3-5d9daf8f801a","Type":"ContainerStarted","Data":"ef00706d869742505e88aaf2763041153abb3cab3d54f758417fdd959077025d"} Jul 21 01:53:35.719538 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.719516 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9289p" event={"ID":"0f019e58-fba6-424e-8878-3aba3b3f1e01","Type":"ContainerStarted","Data":"c26dbe444b3322160815b6dcb00e679804139c0212f2af196be3111c0abfcc73"} Jul 21 01:53:35.720693 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.720669 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" event={"ID":"e80b8395-410e-409c-8e77-7f671ccb99d7","Type":"ContainerStarted","Data":"04c012edd9db3acfa7406df3cf55db48c8334a63e82070e3003f490a8ad20aee"} Jul 21 01:53:35.721816 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.721794 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lf6nt" event={"ID":"67d36da6-d97f-400d-bb0d-2d8986690d0c","Type":"ContainerStarted","Data":"dbfa289c40ea7287143fbb788ccc451dfde565a5eb87e59bfcb166cd70470605"} Jul 21 01:53:35.723011 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:35.722989 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9klcj" event={"ID":"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a","Type":"ContainerStarted","Data":"a450c27494c875bf0816e5934a0f39d0ae6a205059342d648c057725867fab5b"} Jul 21 01:53:36.248620 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.248560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:36.248620 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.248618 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:36.248879 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.248804 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:36.248879 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.248869 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:38.248849277 +0000 UTC m=+6.065883909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:36.249330 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.249272 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:36.249330 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.249327 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:53:38.249309479 +0000 UTC m=+6.066344104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:36.349020 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.348981 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:36.349239 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.349217 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:53:36.349322 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.349246 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:53:36.349322 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.349261 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:36.349322 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.349317 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:38.349297556 +0000 UTC m=+6.166332178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:36.660077 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.659989 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 01:48:34 +0000 UTC" deadline="2028-03-16 09:29:59.951102945 +0000 UTC" Jul 21 01:53:36.660077 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.660032 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14503h36m23.291076188s" Jul 21 01:53:36.702320 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.702290 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:36.702507 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.702439 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:36.702758 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.702586 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:36.706437 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.703139 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:36.706437 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:36.703255 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:36.706437 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:36.703388 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:37.753969 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:37.753723 2561 generic.go:356] "Generic (PLEG): container finished" podID="d3c5fff1526cc232660f7475ed460e6b" containerID="d1d6037dc142e0b1829a632bfd2f0d0ba6f25181e846b90f07975b104bf4c15d" exitCode=0 Jul 21 01:53:37.754598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:37.754039 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" event={"ID":"d3c5fff1526cc232660f7475ed460e6b","Type":"ContainerDied","Data":"d1d6037dc142e0b1829a632bfd2f0d0ba6f25181e846b90f07975b104bf4c15d"} Jul 21 01:53:37.780990 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:37.780957 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" event={"ID":"cf74ce26f0de05ee8701a049d7ed8096","Type":"ContainerStarted","Data":"f9c312686bacac4a05d213e69e4df3e92683d872e915a5894792360d9997a930"} Jul 21 01:53:38.267531 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.266640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:38.267531 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.266701 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:38.267531 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.266863 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:38.267531 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.266927 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:42.266906517 +0000 UTC m=+10.083941135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:38.267531 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.267347 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:38.267531 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.267402 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:53:42.267385424 +0000 UTC m=+10.084420041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:38.368689 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.368027 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:38.368689 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.368202 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:53:38.368689 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.368225 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:53:38.368689 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.368238 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:38.368689 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.368295 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:42.368275356 +0000 UTC m=+10.185309979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:38.702670 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.702566 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:38.702849 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.702712 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:38.703162 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.703143 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:38.703300 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.703270 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:38.703384 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.703347 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:38.703497 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:38.703402 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:38.790551 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.790485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" event={"ID":"d3c5fff1526cc232660f7475ed460e6b","Type":"ContainerStarted","Data":"6f2452a2d2c018d93dfa888060b749c30ae1e3024afa9aff8d17003c2dca0278"} Jul 21 01:53:38.809199 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:38.808793 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-168.ec2.internal" podStartSLOduration=4.808771487 podStartE2EDuration="4.808771487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:37.796131285 +0000 UTC m=+5.613165929" watchObservedRunningTime="2026-07-21 01:53:38.808771487 +0000 UTC m=+6.625806126" Jul 21 01:53:40.706524 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:40.706488 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:40.707027 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:40.706621 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:40.707027 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:40.707024 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:40.707147 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:40.707128 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:40.707198 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:40.707186 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:40.707299 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:40.707254 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:42.299473 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:42.299426 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:42.299473 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:42.299476 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:42.299933 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.299586 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:42.299933 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.299607 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:42.299933 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.299655 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:53:50.299633367 +0000 UTC m=+18.116667997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:42.299933 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.299672 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:50.299664121 +0000 UTC m=+18.116698736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:42.400098 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:42.400060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:42.400267 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.400196 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:53:42.400267 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.400221 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:53:42.400267 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.400233 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:42.400441 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.400295 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:53:50.40027642 +0000 UTC m=+18.217311041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:42.703393 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:42.702950 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:42.703393 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:42.703045 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:42.703393 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.703056 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:42.703393 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:42.703170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:42.703393 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.703164 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:42.703393 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:42.703265 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:44.702694 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:44.702421 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:44.702694 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:44.702421 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:44.702694 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:44.702534 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:44.702694 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:44.702637 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:44.702694 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:44.702436 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:44.703397 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:44.702765 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:46.702658 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.702630 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:46.702658 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.702646 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:46.703093 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:46.702751 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:46.703093 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:46.702866 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:46.703093 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.702919 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:46.703093 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:46.702989 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:46.809180 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.809145 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" event={"ID":"e80b8395-410e-409c-8e77-7f671ccb99d7","Type":"ContainerStarted","Data":"48435148a2b2ae39a75abf828b91aaf190ae252f32d080dfba26a76dd5b41b85"} Jul 21 01:53:46.810679 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.810647 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lf6nt" event={"ID":"67d36da6-d97f-400d-bb0d-2d8986690d0c","Type":"ContainerStarted","Data":"16dcb02d2a6b03caa1231e0eac8e8b3e6dcbfbc187cee7df8fd511ebed8447cd"} Jul 21 01:53:46.812304 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.812281 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerStarted","Data":"152a75fb7808347554205752494cac9c91afb874da468c07cd4fadfc10165e3e"} Jul 21 01:53:46.813894 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.813865 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d49sq" event={"ID":"35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c","Type":"ContainerStarted","Data":"94b94b79e6be7118b19612e6821e65f01594cf458e905e7f5dac28bfd54c5469"} Jul 21 01:53:46.815361 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.815334 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2pmdj" event={"ID":"bde4cbaf-9131-4510-81a3-5d9daf8f801a","Type":"ContainerStarted","Data":"f5eebbca75b11e5973ae99a5d1e5615531040e5da547a7ec59cb49d4166c0aee"} Jul 21 01:53:46.824127 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.824084 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-168.ec2.internal" podStartSLOduration=12.824073577 podStartE2EDuration="12.824073577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:38.809123951 +0000 UTC m=+6.626158591" watchObservedRunningTime="2026-07-21 01:53:46.824073577 +0000 UTC m=+14.641108214" Jul 21 01:53:46.824712 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.824684 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lf6nt" podStartSLOduration=2.657958523 podStartE2EDuration="13.824674875s" totalImagesPullingTime="11.166716352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:46.824017536 +0000 UTC m=+14.641052177" watchObservedRunningTime="2026-07-21 01:53:46.824674875 +0000 UTC m=+14.641709515" Jul 21 01:53:46.836875 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.836833 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2pmdj" podStartSLOduration=2.676599897 podStartE2EDuration="13.836816767s" totalImagesPullingTime="11.16021687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:46.836757618 +0000 UTC m=+14.653792257" watchObservedRunningTime="2026-07-21 01:53:46.836816767 +0000 UTC m=+14.653851405" Jul 21 01:53:46.850287 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:46.850254 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d49sq" podStartSLOduration=2.657875611 podStartE2EDuration="13.850242099s" totalImagesPullingTime="11.192366488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:46.849824773 +0000 UTC m=+14.666859411" watchObservedRunningTime="2026-07-21 01:53:46.850242099 +0000 UTC m=+14.667276738" Jul 21 01:53:48.396677 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.396640 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:48.397623 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.397598 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:48.702516 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.702447 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:48.702516 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.702451 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:48.702726 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:48.702573 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:48.702726 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.702607 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:48.702823 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:48.702716 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:48.702887 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:48.702815 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:48.820778 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.820716 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9289p" event={"ID":"0f019e58-fba6-424e-8878-3aba3b3f1e01","Type":"ContainerStarted","Data":"1439c10a1e1cead5127461c58642bbc96ae0db5f12ff80752a4dbb028a6f47b0"} Jul 21 01:53:48.820949 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.820910 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:48.821352 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.821329 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2pmdj" Jul 21 01:53:48.834923 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:48.834885 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9289p" podStartSLOduration=4.696882415 podStartE2EDuration="15.834871771s" totalImagesPullingTime="11.137989356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:48.834655399 +0000 UTC m=+16.651690038" watchObservedRunningTime="2026-07-21 01:53:48.834871771 +0000 UTC m=+16.651906409" Jul 21 01:53:50.362445 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.362236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:50.362935 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.362497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:50.362935 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.362381 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:50.362935 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.362593 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:54:06.362568559 +0000 UTC m=+34.179603189 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:53:50.362935 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.362600 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:50.362935 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.362649 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:54:06.362634373 +0000 UTC m=+34.179668989 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:53:50.463667 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.463600 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:50.463820 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.463787 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:53:50.463820 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.463814 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:53:50.463931 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.463827 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:50.463931 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.463898 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:54:06.463877912 +0000 UTC m=+34.280912545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:53:50.702135 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.702102 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:50.702135 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.702136 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:50.702356 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.702105 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:50.702356 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.702231 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:50.702356 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.702330 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:50.702534 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:50.702429 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:50.826046 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.825969 2561 generic.go:356] "Generic (PLEG): container finished" podID="e732d16d-8b81-453f-b166-bf540e40195a" containerID="152a75fb7808347554205752494cac9c91afb874da468c07cd4fadfc10165e3e" exitCode=0 Jul 21 01:53:50.826196 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:50.826045 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerDied","Data":"152a75fb7808347554205752494cac9c91afb874da468c07cd4fadfc10165e3e"} Jul 21 01:53:52.702836 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:52.702809 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:52.703539 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:52.702811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:52.703539 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:52.702930 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:52.703539 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:52.702923 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:52.703539 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:52.703010 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:52.703539 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:52.703080 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:54.702738 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:54.702709 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:54.703170 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:54.702709 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:54.703170 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:54.702830 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:54.703170 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:54.702918 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:54.703170 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:54.702709 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:54.703170 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:54.703034 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:55.051625 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.051379 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 01:53:55.683798 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.683687 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T01:53:55.051610136Z","UUID":"7f5fa6b5-7034-4d04-a541-1978f8d7896b","Handler":null,"Name":"","Endpoint":""} Jul 21 01:53:55.685697 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.685676 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 01:53:55.685818 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.685722 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 01:53:55.839789 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.839575 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" event={"ID":"e80b8395-410e-409c-8e77-7f671ccb99d7","Type":"ContainerStarted","Data":"1780697a4f538e2a1e32ecbef6c787109de116b9e5f40ad54c92fdd8de05d72c"} Jul 21 01:53:55.841278 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.841236 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9klcj" event={"ID":"9d67daaf-37d0-4b6a-8f63-fba0e4e51e8a","Type":"ContainerStarted","Data":"354450a6febe56e65ab2b22c048e3281c1a0ddc317dd622db314d6b9cfdffbf1"} Jul 21 01:53:55.845196 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845168 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 01:53:55.845660 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845622 2561 generic.go:356] "Generic (PLEG): container finished" podID="9dbab70a-e2fa-4489-b191-4eed783b55a9" containerID="53c003bfffa190be6d00fe65b05271e850501255c2b0ddf8d33656e43e511a91" exitCode=1 Jul 21 01:53:55.845777 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845687 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"f8e6808bde8143511f5cc6f77937cd5002df234b54cd25c0524c1f926a4b04f5"} Jul 21 01:53:55.845777 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845773 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"b0261777024e998b0e0b987632df7ad9beeac812e952f44b1a06c748c5ac6162"} Jul 21 01:53:55.845894 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845820 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"db636f55a5f8d731ca07c8438f237f506a3207745351cd0e5694cd8fa3cc48ab"} Jul 21 01:53:55.845894 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845866 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"a4b7330204fe8b3bb693e80ce9c31df14da5fb377d6c7b57b9fd7268a093f918"} Jul 21 01:53:55.846000 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845921 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerDied","Data":"53c003bfffa190be6d00fe65b05271e850501255c2b0ddf8d33656e43e511a91"} Jul 21 01:53:55.846000 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.845969 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"2715e0016f753d76c3723ca99f040026b018812e5acb42767ad2ace39953d0aa"} Jul 21 01:53:55.847252 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.847229 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" event={"ID":"883b0920-9cd4-4e33-9fa0-2de1dfb79720","Type":"ContainerStarted","Data":"29dbe28e86eee12a3fafa9051442c6656a602337f7c083a0f68c6053a6c9aa95"} Jul 21 01:53:55.858572 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.858526 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9klcj" podStartSLOduration=3.035694827 podStartE2EDuration="22.858515706s" totalImagesPullingTime="19.822820879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:55.858031083 +0000 UTC m=+23.675065721" watchObservedRunningTime="2026-07-21 01:53:55.858515706 +0000 UTC m=+23.675550343" Jul 21 01:53:55.872436 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:55.872382 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2mwl5" podStartSLOduration=11.434904923 podStartE2EDuration="22.872365854s" totalImagesPullingTime="11.437460931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:55.871641703 +0000 UTC m=+23.688676342" watchObservedRunningTime="2026-07-21 01:53:55.872365854 +0000 UTC m=+23.689400492" Jul 21 01:53:56.703097 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:56.703019 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:56.703288 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:56.703028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:56.703288 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:56.703147 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:56.703288 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:56.703028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:56.703288 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:56.703181 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:56.703288 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:56.703261 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:53:56.850894 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:56.850856 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" event={"ID":"e80b8395-410e-409c-8e77-7f671ccb99d7","Type":"ContainerStarted","Data":"05a171f8252bf681b0decbe948826dc54b3e1ce85ab3a6fadebda853f208f966"} Jul 21 01:53:56.868895 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:56.868853 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9k8j" podStartSLOduration=2.995832004 podStartE2EDuration="23.868840088s" totalImagesPullingTime="20.873008084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:53:56.868765057 +0000 UTC m=+24.685799694" watchObservedRunningTime="2026-07-21 01:53:56.868840088 +0000 UTC m=+24.685874724" Jul 21 01:53:57.856801 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:57.856775 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 01:53:57.857341 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:57.857306 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"2882da84d41f353bb4acf29d3043219c07ed950703c70fb927cdddc12e6d6706"} Jul 21 01:53:58.702615 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:58.702584 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:53:58.702615 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:58.702598 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:53:58.702835 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:53:58.702595 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:53:58.702835 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:58.702697 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:53:58.702835 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:58.702806 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:53:58.702979 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:53:58.702899 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:54:00.703020 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.702814 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:00.703759 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.702814 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:00.703759 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:00.703079 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:54:00.703759 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.702828 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:00.703759 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:00.703120 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:54:00.703759 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:00.703190 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:54:00.867435 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.867396 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 01:54:00.867785 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.867765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"1e35be804d7244c6bc428ba7c404a94e67112429bbbf7d4e7e1d4e830f32b2bb"} Jul 21 01:54:00.867977 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.867961 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:54:00.868170 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.868154 2561 scope.go:117] "RemoveContainer" containerID="53c003bfffa190be6d00fe65b05271e850501255c2b0ddf8d33656e43e511a91" Jul 21 01:54:00.869671 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.869640 2561 generic.go:356] "Generic (PLEG): container finished" podID="e732d16d-8b81-453f-b166-bf540e40195a" containerID="e2561c21cbf7fdbd9c96392e1e639f55b94c019944d4d9ad6f0ae16f21cf23fa" exitCode=0 Jul 21 01:54:00.869757 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.869683 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerDied","Data":"e2561c21cbf7fdbd9c96392e1e639f55b94c019944d4d9ad6f0ae16f21cf23fa"} Jul 21 01:54:00.883810 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:00.883792 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:54:01.878691 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:01.878660 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 01:54:01.879190 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:01.879152 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" event={"ID":"9dbab70a-e2fa-4489-b191-4eed783b55a9","Type":"ContainerStarted","Data":"511d597955e39d0fd36f59e79dc3f0c9bc0cc052941970fc7e4fdc419ad5d863"} Jul 21 01:54:01.879645 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:01.879625 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:54:01.879789 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:01.879693 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:54:01.895644 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:01.895615 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:54:01.905352 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:01.905308 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" podStartSLOduration=8.961914185 podStartE2EDuration="28.905293427s" totalImagesPullingTime="19.943379242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:54:01.904323782 +0000 UTC m=+29.721358421" watchObservedRunningTime="2026-07-21 01:54:01.905293427 +0000 UTC m=+29.722328065" Jul 21 01:54:02.140186 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.140139 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d6brs"] Jul 21 01:54:02.140290 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.140275 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:02.140460 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:02.140424 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:54:02.144111 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.144081 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wt28r"] Jul 21 01:54:02.144207 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.144164 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:02.144265 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:02.144245 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:54:02.156447 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.156421 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m64bz"] Jul 21 01:54:02.156536 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.156513 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:02.156599 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:02.156584 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:54:02.882962 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.882929 2561 generic.go:356] "Generic (PLEG): container finished" podID="e732d16d-8b81-453f-b166-bf540e40195a" containerID="4b82acf868cfef3876c7df33df215495bf32f9d78989274718faf1a728736d0e" exitCode=0 Jul 21 01:54:02.883340 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:02.883008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerDied","Data":"4b82acf868cfef3876c7df33df215495bf32f9d78989274718faf1a728736d0e"} Jul 21 01:54:03.703095 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:03.703061 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:03.703232 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:03.703172 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:03.703232 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:03.703181 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:54:03.703323 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:03.703291 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:54:03.703323 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:03.703181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:03.703467 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:03.703442 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:54:04.889071 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:04.889036 2561 generic.go:356] "Generic (PLEG): container finished" podID="e732d16d-8b81-453f-b166-bf540e40195a" containerID="8390bfc395051d4974eeccb459598188ea7b46cc98729b831b1745656ac81b71" exitCode=0 Jul 21 01:54:04.889443 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:04.889079 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerDied","Data":"8390bfc395051d4974eeccb459598188ea7b46cc98729b831b1745656ac81b71"} Jul 21 01:54:05.702687 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:05.702657 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:05.702849 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:05.702658 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:05.702849 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:05.702743 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:54:05.702849 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:05.702662 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:05.702983 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:05.702818 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:54:05.702983 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:05.702903 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:54:06.386173 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:06.385991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:06.386173 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.386158 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 01:54:06.386792 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:06.386190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:06.386792 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.386230 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret podName:40ff25a9-41ca-4f8c-aa44-f963d44e5a3b nodeName:}" failed. No retries permitted until 2026-07-21 01:54:38.386213173 +0000 UTC m=+66.203247807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret") pod "global-pull-secret-syncer-wt28r" (UID: "40ff25a9-41ca-4f8c-aa44-f963d44e5a3b") : object "kube-system"/"original-pull-secret" not registered Jul 21 01:54:06.386792 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.386281 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:54:06.386792 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.386333 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:54:38.386317036 +0000 UTC m=+66.203351652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 01:54:06.486657 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:06.486587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:06.486786 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.486729 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 01:54:06.486786 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.486750 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 01:54:06.486786 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.486759 2561 projected.go:196] Error preparing data for projected volume kube-api-access-ddxgd for pod openshift-network-diagnostics/network-check-target-m64bz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:54:06.486882 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:06.486803 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd podName:62cdd62a-d8ec-4114-a45b-eaf13950cd97 nodeName:}" failed. No retries permitted until 2026-07-21 01:54:38.48679064 +0000 UTC m=+66.303825261 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddxgd" (UniqueName: "kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd") pod "network-check-target-m64bz" (UID: "62cdd62a-d8ec-4114-a45b-eaf13950cd97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 01:54:07.702865 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:07.702835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:07.703303 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:07.702840 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:07.703303 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:07.702940 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m64bz" podUID="62cdd62a-d8ec-4114-a45b-eaf13950cd97" Jul 21 01:54:07.703303 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:07.702970 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:07.703303 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:07.703060 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:54:07.703303 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:07.703119 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wt28r" podUID="40ff25a9-41ca-4f8c-aa44-f963d44e5a3b" Jul 21 01:54:08.469554 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.469486 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-168.ec2.internal" event="NodeReady" Jul 21 01:54:08.469681 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.469574 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 01:54:08.503227 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.503189 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9"] Jul 21 01:54:08.506676 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.506657 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d4ff45c9b-65mlr"] Jul 21 01:54:08.506811 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.506792 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:08.508588 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.508552 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 01:54:08.508712 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.508643 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 01:54:08.508956 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.508941 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l4zpp\"" Jul 21 01:54:08.509888 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.509871 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.511703 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.511682 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k8zlf\"" Jul 21 01:54:08.512245 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.512226 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 01:54:08.512453 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.512340 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 01:54:08.512453 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.512393 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 01:54:08.518138 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.518085 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9"] Jul 21 01:54:08.518788 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.518762 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7hq8m"] Jul 21 01:54:08.519118 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.519101 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 01:54:08.524661 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.524641 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d4ff45c9b-65mlr"] Jul 21 01:54:08.524728 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.524680 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nbst9"] Jul 21 01:54:08.524902 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.524876 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.527647 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.527498 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 01:54:08.527647 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.527556 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 01:54:08.527945 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.527925 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cbx74\"" Jul 21 01:54:08.528155 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.528135 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.530372 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.530353 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 01:54:08.531877 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.531856 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 01:54:08.532263 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.532244 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g2jzl\"" Jul 21 01:54:08.534452 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.533872 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 01:54:08.534452 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.534232 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hq8m"] Jul 21 01:54:08.534957 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.534942 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nbst9"] Jul 21 01:54:08.604520 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:08.604635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604527 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-certificates\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604548 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42ad014a-085f-4801-9869-6f03c196609a-config-volume\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.604635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604572 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42ad014a-085f-4801-9869-6f03c196609a-tmp-dir\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.604635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604595 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604614 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-installation-pull-secrets\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-bound-sa-token\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604646 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk9vb\" (UniqueName: \"kubernetes.io/projected/42ad014a-085f-4801-9869-6f03c196609a-kube-api-access-pk9vb\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e69a0874-21c0-4528-aec5-febde2de9500-cert\") pod \"ingress-canary-nbst9\" (UID: \"e69a0874-21c0-4528-aec5-febde2de9500\") " pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604752 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604793 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q878\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-kube-api-access-9q878\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3e3d345-33d7-47a0-9128-3776ddef87bc-ca-trust-extracted\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604834 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604851 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-trusted-ca\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-image-registry-private-configuration\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.604928 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.604893 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6nv2\" (UniqueName: \"kubernetes.io/projected/e69a0874-21c0-4528-aec5-febde2de9500-kube-api-access-w6nv2\") pod \"ingress-canary-nbst9\" (UID: \"e69a0874-21c0-4528-aec5-febde2de9500\") " pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.705363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705339 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-image-registry-private-configuration\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705375 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6nv2\" (UniqueName: \"kubernetes.io/projected/e69a0874-21c0-4528-aec5-febde2de9500-kube-api-access-w6nv2\") pod \"ingress-canary-nbst9\" (UID: \"e69a0874-21c0-4528-aec5-febde2de9500\") " pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705453 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-certificates\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705475 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42ad014a-085f-4801-9869-6f03c196609a-config-volume\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705500 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42ad014a-085f-4801-9869-6f03c196609a-tmp-dir\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705553 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-installation-pull-secrets\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-bound-sa-token\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705599 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk9vb\" (UniqueName: \"kubernetes.io/projected/42ad014a-085f-4801-9869-6f03c196609a-kube-api-access-pk9vb\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.705622 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e69a0874-21c0-4528-aec5-febde2de9500-cert\") pod \"ingress-canary-nbst9\" (UID: \"e69a0874-21c0-4528-aec5-febde2de9500\") " pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.705759 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.705775 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:08.705817 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.705824 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:54:09.205807936 +0000 UTC m=+37.022842566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:08.706464 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706353 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-certificates\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.706515 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706488 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/42ad014a-085f-4801-9869-6f03c196609a-config-volume\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.706584 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706564 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:08.706701 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706683 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42ad014a-085f-4801-9869-6f03c196609a-tmp-dir\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.706747 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706724 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:08.706790 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q878\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-kube-api-access-9q878\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.706837 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706803 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3e3d345-33d7-47a0-9128-3776ddef87bc-ca-trust-extracted\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.706837 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.706932 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.706854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-trusted-ca\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.707472 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.707271 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:08.707472 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.707327 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:54:09.207308783 +0000 UTC m=+37.024343407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:08.707472 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.707373 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3e3d345-33d7-47a0-9128-3776ddef87bc-ca-trust-extracted\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.707472 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.707395 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:08.707472 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:08.707449 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:54:09.207436269 +0000 UTC m=+37.024470888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:08.707793 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.707736 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-trusted-ca\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.710129 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.710104 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-image-registry-private-configuration\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.710290 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.710146 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e69a0874-21c0-4528-aec5-febde2de9500-cert\") pod \"ingress-canary-nbst9\" (UID: \"e69a0874-21c0-4528-aec5-febde2de9500\") " pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.710290 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.710157 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-installation-pull-secrets\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.727537 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.727486 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-bound-sa-token\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.727649 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.727632 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q878\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-kube-api-access-9q878\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:08.727691 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.727648 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk9vb\" (UniqueName: \"kubernetes.io/projected/42ad014a-085f-4801-9869-6f03c196609a-kube-api-access-pk9vb\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:08.727772 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.727758 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6nv2\" (UniqueName: \"kubernetes.io/projected/e69a0874-21c0-4528-aec5-febde2de9500-kube-api-access-w6nv2\") pod \"ingress-canary-nbst9\" (UID: \"e69a0874-21c0-4528-aec5-febde2de9500\") " pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.845735 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.845706 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nbst9" Jul 21 01:54:08.983598 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:08.983532 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nbst9"] Jul 21 01:54:08.989328 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:54:08.989292 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69a0874_21c0_4528_aec5_febde2de9500.slice/crio-a08fd6fc337add6a4d30db3d291ad160ada18cf5c1bd465f519aa376d354c394 WatchSource:0}: Error finding container a08fd6fc337add6a4d30db3d291ad160ada18cf5c1bd465f519aa376d354c394: Status 404 returned error can't find the container with id a08fd6fc337add6a4d30db3d291ad160ada18cf5c1bd465f519aa376d354c394 Jul 21 01:54:09.211201 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.211170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:09.211339 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.211212 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:09.211339 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.211255 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:09.211339 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211316 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:09.211339 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211336 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:09.211540 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211386 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:54:10.211370791 +0000 UTC m=+38.028405407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:09.211540 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211414 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:54:10.211395185 +0000 UTC m=+38.028429801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:09.211540 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211400 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:09.211540 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211436 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:09.211540 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:09.211485 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:54:10.211470052 +0000 UTC m=+38.028504669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:09.702829 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.702799 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:09.703021 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.702859 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:09.703021 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.702952 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:09.705181 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.705146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 01:54:09.705338 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.705180 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 01:54:09.705338 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.705187 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 01:54:09.706227 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.705634 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 01:54:09.706227 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.705699 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g586n\"" Jul 21 01:54:09.706227 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.705732 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8gmrt\"" Jul 21 01:54:09.904126 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:09.904079 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nbst9" event={"ID":"e69a0874-21c0-4528-aec5-febde2de9500","Type":"ContainerStarted","Data":"a08fd6fc337add6a4d30db3d291ad160ada18cf5c1bd465f519aa376d354c394"} Jul 21 01:54:10.219894 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:10.219862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:10.220066 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:10.219962 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:10.220066 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:10.219998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:10.220066 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220032 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:10.220066 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220055 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:10.220337 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220112 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:10.220337 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220124 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:54:12.220102245 +0000 UTC m=+40.037136879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:10.220337 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220170 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:54:12.220154256 +0000 UTC m=+40.037188893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:10.220337 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220184 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:10.220337 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:10.220243 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:54:12.220227879 +0000 UTC m=+40.037262500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:11.913312 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:11.913272 2561 generic.go:356] "Generic (PLEG): container finished" podID="e732d16d-8b81-453f-b166-bf540e40195a" containerID="ec9267553639932ef6cd2c24d45bde9ed0f1026f7de8a4923c5adead85e67900" exitCode=0 Jul 21 01:54:11.913715 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:11.913339 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerDied","Data":"ec9267553639932ef6cd2c24d45bde9ed0f1026f7de8a4923c5adead85e67900"} Jul 21 01:54:12.237630 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.237564 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:12.237630 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.237605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.237640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237735 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237752 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237763 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237735 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237800 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:54:16.237781774 +0000 UTC m=+44.054816396 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:12.237816 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237816 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:54:16.237807884 +0000 UTC m=+44.054842501 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:12.238036 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:12.237828 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:54:16.237821492 +0000 UTC m=+44.054856110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:12.824344 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.824310 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:12.904675 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.904639 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:12.919606 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.919568 2561 generic.go:356] "Generic (PLEG): container finished" podID="e732d16d-8b81-453f-b166-bf540e40195a" containerID="1aacb37dea7f4de432c26f235e815ea976cf87da6b33d200318bfcf571f1c6d8" exitCode=0 Jul 21 01:54:12.920006 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:12.919616 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerDied","Data":"1aacb37dea7f4de432c26f235e815ea976cf87da6b33d200318bfcf571f1c6d8"} Jul 21 01:54:13.925569 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:13.925481 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" event={"ID":"e732d16d-8b81-453f-b166-bf540e40195a","Type":"ContainerStarted","Data":"d6f5ab041b2831e3068327f474046515b81ee128aaf3f1019ff2a0adb0eaac68"} Jul 21 01:54:13.926934 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:13.926908 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nbst9" event={"ID":"e69a0874-21c0-4528-aec5-febde2de9500","Type":"ContainerStarted","Data":"9cbdd1fd52b122220e326cc2ff5616be99e75c3a9d63de08e2603dc773f904ef"} Jul 21 01:54:13.950340 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:13.950286 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d9tm7" podStartSLOduration=11.776467593 podStartE2EDuration="40.950267443s" totalImagesPullingTime="29.17379985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:54:13.9488682 +0000 UTC m=+41.765902839" watchObservedRunningTime="2026-07-21 01:54:13.950267443 +0000 UTC m=+41.767302083" Jul 21 01:54:13.964656 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:13.964570 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nbst9" podStartSLOduration=1.287489865 podStartE2EDuration="5.964553882s" totalImagesPullingTime="4.677064017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:54:13.964173483 +0000 UTC m=+41.781208133" watchObservedRunningTime="2026-07-21 01:54:13.964553882 +0000 UTC m=+41.781588521" Jul 21 01:54:14.079184 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:14.079150 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:14.136142 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:14.136110 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:16.267037 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:16.267002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:16.267046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:16.267079 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267173 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267182 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267213 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267187 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267251 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:54:24.267231623 +0000 UTC m=+52.084266239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267274 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:54:24.267262113 +0000 UTC m=+52.084296729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:16.267499 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:16.267291 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:54:24.26728268 +0000 UTC m=+52.084317298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:16.499103 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:16.499070 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:16.550974 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:16.550911 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:16.599379 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:16.599349 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:17.313931 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:17.313890 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:17.414156 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:17.414101 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:19.057645 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:19.057612 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:19.158795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:19.158759 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:19.217780 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:19.217755 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:20.173840 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:20.173809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:20.274779 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:20.274749 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:24.327878 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:24.327842 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:24.327913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:24.327964 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328006 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328030 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328042 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328010 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328083 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:54:40.328067093 +0000 UTC m=+68.145101741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328101 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:54:40.328092084 +0000 UTC m=+68.145126701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:24.328255 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:24.328114 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:54:40.328107153 +0000 UTC m=+68.145141789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:28.140940 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:28.140899 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:28.200568 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:28.200532 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:31.327560 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:31.327525 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:31.756978 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:31.756945 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:32.379227 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:32.379192 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:32.480073 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:32.480030 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:33.903262 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:33.903234 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qfcwj" Jul 21 01:54:34.059736 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:34.059700 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:34.124041 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:34.124015 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:34.405437 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:34.405396 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:34.480172 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:34.480138 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:35.149981 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:35.149950 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:35.248314 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:35.248291 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:35.420288 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:35.420219 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:35.475061 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:35.475033 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:35.764605 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:35.764516 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:35.821442 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:35.821393 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:38.418523 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.418483 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:38.418523 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.418528 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:54:38.420677 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.420655 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 01:54:38.420809 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.420705 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 01:54:38.429265 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:38.429248 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 01:54:38.429355 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:38.429311 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:55:42.429291633 +0000 UTC m=+130.246326266 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : secret "metrics-daemon-secret" not found Jul 21 01:54:38.432205 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.432187 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40ff25a9-41ca-4f8c-aa44-f963d44e5a3b-original-pull-secret\") pod \"global-pull-secret-syncer-wt28r\" (UID: \"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b\") " pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:38.519478 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.519446 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:38.521706 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.521691 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 01:54:38.531658 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.531637 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wt28r" Jul 21 01:54:38.532286 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.532272 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 01:54:38.543259 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.543239 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddxgd\" (UniqueName: \"kubernetes.io/projected/62cdd62a-d8ec-4114-a45b-eaf13950cd97-kube-api-access-ddxgd\") pod \"network-check-target-m64bz\" (UID: \"62cdd62a-d8ec-4114-a45b-eaf13950cd97\") " pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:38.669949 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.669888 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wt28r"] Jul 21 01:54:38.672853 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:54:38.672829 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ff25a9_41ca_4f8c_aa44_f963d44e5a3b.slice/crio-f27aee9b29e22827223699eb62461d575d6c73a33c0274bbab8a2f6dba279556 WatchSource:0}: Error finding container f27aee9b29e22827223699eb62461d575d6c73a33c0274bbab8a2f6dba279556: Status 404 returned error can't find the container with id f27aee9b29e22827223699eb62461d575d6c73a33c0274bbab8a2f6dba279556 Jul 21 01:54:38.821974 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.821950 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g586n\"" Jul 21 01:54:38.830620 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.830603 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:38.944213 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.944149 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m64bz"] Jul 21 01:54:38.946884 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:54:38.946856 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cdd62a_d8ec_4114_a45b_eaf13950cd97.slice/crio-b41443013154b483ed8f94161096449f2c8cff2723735d8631cc0b1f370b2e0b WatchSource:0}: Error finding container b41443013154b483ed8f94161096449f2c8cff2723735d8631cc0b1f370b2e0b: Status 404 returned error can't find the container with id b41443013154b483ed8f94161096449f2c8cff2723735d8631cc0b1f370b2e0b Jul 21 01:54:38.980741 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.980711 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wt28r" event={"ID":"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b","Type":"ContainerStarted","Data":"f27aee9b29e22827223699eb62461d575d6c73a33c0274bbab8a2f6dba279556"} Jul 21 01:54:38.981768 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:38.981748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m64bz" event={"ID":"62cdd62a-d8ec-4114-a45b-eaf13950cd97","Type":"ContainerStarted","Data":"b41443013154b483ed8f94161096449f2c8cff2723735d8631cc0b1f370b2e0b"} Jul 21 01:54:40.333202 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:40.333164 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:40.333220 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:40.333249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333331 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333351 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333374 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333375 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333430 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:55:12.333399978 +0000 UTC m=+100.150434595 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333458 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:55:12.333438753 +0000 UTC m=+100.150473386 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:54:40.333644 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:54:40.333478 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:55:12.333468522 +0000 UTC m=+100.150503179 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:54:42.994705 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:42.994613 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m64bz" event={"ID":"62cdd62a-d8ec-4114-a45b-eaf13950cd97","Type":"ContainerStarted","Data":"bbdf4663e9291b35aa150a463fb98ac2dc6a755fb7ce06ceb67e1509f6da74d4"} Jul 21 01:54:42.994705 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:42.994700 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:54:43.009086 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:43.008857 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m64bz" podStartSLOduration=66.30485278 podStartE2EDuration="1m10.008839517s" totalImagesPullingTime="3.703986737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:54:43.008469817 +0000 UTC m=+70.825504456" watchObservedRunningTime="2026-07-21 01:54:43.008839517 +0000 UTC m=+70.825874146" Jul 21 01:54:45.001288 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:45.001253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wt28r" event={"ID":"40ff25a9-41ca-4f8c-aa44-f963d44e5a3b","Type":"ContainerStarted","Data":"52376f54e09898203124ac030520534802d28b0bd24d69ceebe2ee8bcba2f463"} Jul 21 01:54:45.015469 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:45.015429 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wt28r" podStartSLOduration=66.116859049 podStartE2EDuration="1m12.015398744s" totalImagesPullingTime="5.898539695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:54:45.014532897 +0000 UTC m=+72.831567536" watchObservedRunningTime="2026-07-21 01:54:45.015398744 +0000 UTC m=+72.832433382" Jul 21 01:54:55.399828 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:55.399715 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:55.463077 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:55.463048 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:57.135810 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:57.135772 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:54:57.381702 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:54:57.381671 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:55:06.058622 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:55:06.058577 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:55:12.338491 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:55:12.338448 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:55:12.338491 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:55:12.338499 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:55:12.338526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338592 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338623 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338634 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338650 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338696 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:56:16.338673002 +0000 UTC m=+164.155707618 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338720 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:56:16.338710887 +0000 UTC m=+164.155745503 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:55:12.338900 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:12.338738 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:56:16.338730685 +0000 UTC m=+164.155765309 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:55:14.000563 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:55:14.000534 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m64bz" Jul 21 01:55:42.441466 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:55:42.441390 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:55:42.442005 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:42.441559 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 01:55:42.442005 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:55:42.441653 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:57:44.441632162 +0000 UTC m=+252.258666796 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : secret "metrics-daemon-secret" not found Jul 21 01:56:11.521450 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:11.521395 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" podUID="d5e054ae-c8cc-41cf-b425-41ed5e1206ff" Jul 21 01:56:11.527631 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:11.527613 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" podUID="e3e3d345-33d7-47a0-9128-3776ddef87bc" Jul 21 01:56:11.541889 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:11.541861 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7hq8m" podUID="42ad014a-085f-4801-9869-6f03c196609a" Jul 21 01:56:12.192877 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:12.192846 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:56:12.193037 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:12.192846 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hq8m" Jul 21 01:56:12.193037 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:12.192851 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:56:12.726531 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:12.726498 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:56:16.368428 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:16.368371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:16.368442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:16.368473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368539 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368563 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368578 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368615 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368632 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 01:58:18.368610727 +0000 UTC m=+286.185645356 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368650 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 01:58:18.368641083 +0000 UTC m=+286.185675702 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:56:16.368911 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:56:16.368666 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:18.368657647 +0000 UTC m=+286.185692264 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:56:26.702920 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:56:26.702842 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:57:28.798087 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:57:28.798052 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:57:44.459374 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:57:44.459333 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:57:44.459836 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:57:44.459486 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 01:57:44.459836 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:57:44.459552 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs podName:956c97dd-8b8b-4dd0-b22b-43110503e7f7 nodeName:}" failed. No retries permitted until 2026-07-21 01:59:46.459536771 +0000 UTC m=+374.276571387 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs") pod "network-metrics-daemon-d6brs" (UID: "956c97dd-8b8b-4dd0-b22b-43110503e7f7") : secret "metrics-daemon-secret" not found Jul 21 01:58:05.077739 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.077661 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz"] Jul 21 01:58:05.080377 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.080352 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.081361 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.081336 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw"] Jul 21 01:58:05.082419 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.082389 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 21 01:58:05.082917 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.082898 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rj5n6\"" Jul 21 01:58:05.083004 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.082934 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 21 01:58:05.083004 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.082962 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 21 01:58:05.083004 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.082934 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 21 01:58:05.084045 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.084022 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.085057 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.085035 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk"] Jul 21 01:58:05.085680 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.085664 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 21 01:58:05.086141 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.086125 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 21 01:58:05.086238 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.086151 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 21 01:58:05.086238 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.086152 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 21 01:58:05.086587 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.086568 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-87bd7\"" Jul 21 01:58:05.087840 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.087729 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.089335 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.089319 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 01:58:05.089532 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.089517 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 01:58:05.089809 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.089794 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ppntw\"" Jul 21 01:58:05.090155 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.090133 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 01:58:05.095662 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.095640 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 01:58:05.097261 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.097240 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz"] Jul 21 01:58:05.098437 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.098415 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw"] Jul 21 01:58:05.110065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.110045 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk"] Jul 21 01:58:05.169457 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.169437 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg"] Jul 21 01:58:05.172247 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.172233 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" Jul 21 01:58:05.174177 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.174151 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 21 01:58:05.174435 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.174397 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kb25x\"" Jul 21 01:58:05.174555 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.174418 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 21 01:58:05.181061 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.181029 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8"] Jul 21 01:58:05.183866 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.183849 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq"] Jul 21 01:58:05.183972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.183956 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.186096 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.186079 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 21 01:58:05.186513 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.186470 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-b52g5\"" Jul 21 01:58:05.186640 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.186625 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg"] Jul 21 01:58:05.186712 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.186699 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" Jul 21 01:58:05.186785 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.186768 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 21 01:58:05.186841 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.186772 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 21 01:58:05.188580 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.188556 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-sqm2z\"" Jul 21 01:58:05.199094 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.199074 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq"] Jul 21 01:58:05.199822 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.199805 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8"] Jul 21 01:58:05.201097 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201077 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxzg5\" (UniqueName: \"kubernetes.io/projected/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-kube-api-access-qxzg5\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.201205 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201107 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84858abb-8a6a-4cc7-8766-f03b2b178381-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.201270 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201206 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1822ea3d-9272-433d-9e6e-ad374d264b40-config\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.201270 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201257 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.201372 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.201372 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201315 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84858abb-8a6a-4cc7-8766-f03b2b178381-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.201372 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201343 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgmv9\" (UniqueName: \"kubernetes.io/projected/1822ea3d-9272-433d-9e6e-ad374d264b40-kube-api-access-xgmv9\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.201603 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201426 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wznqq\" (UniqueName: \"kubernetes.io/projected/84858abb-8a6a-4cc7-8766-f03b2b178381-kube-api-access-wznqq\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.201603 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.201461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1822ea3d-9272-433d-9e6e-ad374d264b40-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.301720 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.301695 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.301829 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.301746 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgmv9\" (UniqueName: \"kubernetes.io/projected/1822ea3d-9272-433d-9e6e-ad374d264b40-kube-api-access-xgmv9\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.301829 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.301792 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wznqq\" (UniqueName: \"kubernetes.io/projected/84858abb-8a6a-4cc7-8766-f03b2b178381-kube-api-access-wznqq\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.301829 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.301819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1822ea3d-9272-433d-9e6e-ad374d264b40-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.301986 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.301958 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7p2c\" (UniqueName: \"kubernetes.io/projected/299a11e5-9614-46c3-8388-eb4adb98dde8-kube-api-access-l7p2c\") pod \"network-check-source-7bc5b6bf85-zjxlq\" (UID: \"299a11e5-9614-46c3-8388-eb4adb98dde8\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" Jul 21 01:58:05.302048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.301999 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzg5\" (UniqueName: \"kubernetes.io/projected/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-kube-api-access-qxzg5\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.302048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302033 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84858abb-8a6a-4cc7-8766-f03b2b178381-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.302144 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7gn\" (UniqueName: \"kubernetes.io/projected/a300cc4c-bd94-4713-9d23-62561345e087-kube-api-access-9z7gn\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.302144 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.302237 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1822ea3d-9272-433d-9e6e-ad374d264b40-config\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.302288 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.302268 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:05.302288 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.302387 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.302327 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls podName:a48756e8-7067-4b95-92cf-e3b7cfde8a8a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:05.802309294 +0000 UTC m=+273.619343916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kgjhk" (UID: "a48756e8-7067-4b95-92cf-e3b7cfde8a8a") : secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:05.302387 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302357 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpwrr\" (UniqueName: \"kubernetes.io/projected/468a2460-be01-417a-a822-e1664b4c121e-kube-api-access-mpwrr\") pod \"volume-data-source-validator-78cf4c6f6-qbswg\" (UID: \"468a2460-be01-417a-a822-e1664b4c121e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" Jul 21 01:58:05.302537 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302400 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84858abb-8a6a-4cc7-8766-f03b2b178381-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.302537 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302463 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.302785 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302765 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1822ea3d-9272-433d-9e6e-ad374d264b40-config\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.302955 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.302937 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84858abb-8a6a-4cc7-8766-f03b2b178381-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.304129 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.304111 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1822ea3d-9272-433d-9e6e-ad374d264b40-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.304236 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.304221 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84858abb-8a6a-4cc7-8766-f03b2b178381-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.309349 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.309323 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxzg5\" (UniqueName: \"kubernetes.io/projected/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-kube-api-access-qxzg5\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.310242 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.310221 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wznqq\" (UniqueName: \"kubernetes.io/projected/84858abb-8a6a-4cc7-8766-f03b2b178381-kube-api-access-wznqq\") pod \"kube-storage-version-migrator-operator-7658d9b749-284bw\" (UID: \"84858abb-8a6a-4cc7-8766-f03b2b178381\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.310318 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.310289 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgmv9\" (UniqueName: \"kubernetes.io/projected/1822ea3d-9272-433d-9e6e-ad374d264b40-kube-api-access-xgmv9\") pod \"service-ca-operator-5bcbc94c78-5wzzz\" (UID: \"1822ea3d-9272-433d-9e6e-ad374d264b40\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.314031 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.314002 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:05.368718 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.368696 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:05.391695 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.391672 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" Jul 21 01:58:05.404088 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.404052 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.404190 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.404114 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpwrr\" (UniqueName: \"kubernetes.io/projected/468a2460-be01-417a-a822-e1664b4c121e-kube-api-access-mpwrr\") pod \"volume-data-source-validator-78cf4c6f6-qbswg\" (UID: \"468a2460-be01-417a-a822-e1664b4c121e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" Jul 21 01:58:05.404253 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.404190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7p2c\" (UniqueName: \"kubernetes.io/projected/299a11e5-9614-46c3-8388-eb4adb98dde8-kube-api-access-l7p2c\") pod \"network-check-source-7bc5b6bf85-zjxlq\" (UID: \"299a11e5-9614-46c3-8388-eb4adb98dde8\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" Jul 21 01:58:05.404253 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.404225 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7gn\" (UniqueName: \"kubernetes.io/projected/a300cc4c-bd94-4713-9d23-62561345e087-kube-api-access-9z7gn\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.404354 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.404253 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 01:58:05.404354 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.404324 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls podName:a300cc4c-bd94-4713-9d23-62561345e087 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:05.904305676 +0000 UTC m=+273.721340295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-n64q8" (UID: "a300cc4c-bd94-4713-9d23-62561345e087") : secret "samples-operator-tls" not found Jul 21 01:58:05.404570 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.404551 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" Jul 21 01:58:05.412102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.412080 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpwrr\" (UniqueName: \"kubernetes.io/projected/468a2460-be01-417a-a822-e1664b4c121e-kube-api-access-mpwrr\") pod \"volume-data-source-validator-78cf4c6f6-qbswg\" (UID: \"468a2460-be01-417a-a822-e1664b4c121e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" Jul 21 01:58:05.412313 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.412292 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7p2c\" (UniqueName: \"kubernetes.io/projected/299a11e5-9614-46c3-8388-eb4adb98dde8-kube-api-access-l7p2c\") pod \"network-check-source-7bc5b6bf85-zjxlq\" (UID: \"299a11e5-9614-46c3-8388-eb4adb98dde8\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" Jul 21 01:58:05.412474 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.412459 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7gn\" (UniqueName: \"kubernetes.io/projected/a300cc4c-bd94-4713-9d23-62561345e087-kube-api-access-9z7gn\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.481011 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.480758 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" Jul 21 01:58:05.497660 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.497629 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" Jul 21 01:58:05.526254 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.526188 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz"] Jul 21 01:58:05.530874 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:05.530844 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1822ea3d_9272_433d_9e6e_ad374d264b40.slice/crio-96f10888d67c4276c85e4b4bf5cdad693368806c8f311031bbd9cef53307999b WatchSource:0}: Error finding container 96f10888d67c4276c85e4b4bf5cdad693368806c8f311031bbd9cef53307999b: Status 404 returned error can't find the container with id 96f10888d67c4276c85e4b4bf5cdad693368806c8f311031bbd9cef53307999b Jul 21 01:58:05.542536 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.542512 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw"] Jul 21 01:58:05.545794 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:05.545763 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84858abb_8a6a_4cc7_8766_f03b2b178381.slice/crio-8c2c50cfca9758a3c7be54422ca6acf938b6211d3f34cec631dc5cc2cc4e1773 WatchSource:0}: Error finding container 8c2c50cfca9758a3c7be54422ca6acf938b6211d3f34cec631dc5cc2cc4e1773: Status 404 returned error can't find the container with id 8c2c50cfca9758a3c7be54422ca6acf938b6211d3f34cec631dc5cc2cc4e1773 Jul 21 01:58:05.629451 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.629364 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq"] Jul 21 01:58:05.631852 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:05.631829 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299a11e5_9614_46c3_8388_eb4adb98dde8.slice/crio-a425fc8600aefb79e234b006839dd816a2817ab81e354995c5ab64a776f8a518 WatchSource:0}: Error finding container a425fc8600aefb79e234b006839dd816a2817ab81e354995c5ab64a776f8a518: Status 404 returned error can't find the container with id a425fc8600aefb79e234b006839dd816a2817ab81e354995c5ab64a776f8a518 Jul 21 01:58:05.642184 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.642159 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg"] Jul 21 01:58:05.645931 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:05.645869 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468a2460_be01_417a_a822_e1664b4c121e.slice/crio-a8c3e9de60242ac30d34cfc068723e5d31bb2d5570a737f0c86a2a2c03d2eb96 WatchSource:0}: Error finding container a8c3e9de60242ac30d34cfc068723e5d31bb2d5570a737f0c86a2a2c03d2eb96: Status 404 returned error can't find the container with id a8c3e9de60242ac30d34cfc068723e5d31bb2d5570a737f0c86a2a2c03d2eb96 Jul 21 01:58:05.807956 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.807915 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:05.808138 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.808075 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:05.808210 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.808150 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls podName:a48756e8-7067-4b95-92cf-e3b7cfde8a8a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:06.808130876 +0000 UTC m=+274.625165498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kgjhk" (UID: "a48756e8-7067-4b95-92cf-e3b7cfde8a8a") : secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:05.908671 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:05.908582 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:05.908814 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.908752 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 01:58:05.908852 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:05.908833 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls podName:a300cc4c-bd94-4713-9d23-62561345e087 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:06.908809007 +0000 UTC m=+274.725843624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-n64q8" (UID: "a300cc4c-bd94-4713-9d23-62561345e087") : secret "samples-operator-tls" not found Jul 21 01:58:06.454355 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.454319 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" event={"ID":"468a2460-be01-417a-a822-e1664b4c121e","Type":"ContainerStarted","Data":"a8c3e9de60242ac30d34cfc068723e5d31bb2d5570a737f0c86a2a2c03d2eb96"} Jul 21 01:58:06.455635 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.455578 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" event={"ID":"84858abb-8a6a-4cc7-8766-f03b2b178381","Type":"ContainerStarted","Data":"8c2c50cfca9758a3c7be54422ca6acf938b6211d3f34cec631dc5cc2cc4e1773"} Jul 21 01:58:06.456886 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.456862 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" event={"ID":"1822ea3d-9272-433d-9e6e-ad374d264b40","Type":"ContainerStarted","Data":"96f10888d67c4276c85e4b4bf5cdad693368806c8f311031bbd9cef53307999b"} Jul 21 01:58:06.458186 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.458162 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" event={"ID":"299a11e5-9614-46c3-8388-eb4adb98dde8","Type":"ContainerStarted","Data":"541c1d43deae36e681553b257e19e95fd1d4d0ffb47286ac029ee9f437ea625c"} Jul 21 01:58:06.458278 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.458205 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" event={"ID":"299a11e5-9614-46c3-8388-eb4adb98dde8","Type":"ContainerStarted","Data":"a425fc8600aefb79e234b006839dd816a2817ab81e354995c5ab64a776f8a518"} Jul 21 01:58:06.483608 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.483558 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zjxlq" podStartSLOduration=1.483540537 podStartE2EDuration="1.483540537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:06.482908577 +0000 UTC m=+274.299943218" watchObservedRunningTime="2026-07-21 01:58:06.483540537 +0000 UTC m=+274.300575176" Jul 21 01:58:06.815375 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.815291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:06.815552 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:06.815439 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:06.815552 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:06.815520 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls podName:a48756e8-7067-4b95-92cf-e3b7cfde8a8a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:08.815497729 +0000 UTC m=+276.632532350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kgjhk" (UID: "a48756e8-7067-4b95-92cf-e3b7cfde8a8a") : secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:06.916211 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:06.916175 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:06.916398 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:06.916386 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 01:58:06.916503 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:06.916464 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls podName:a300cc4c-bd94-4713-9d23-62561345e087 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:08.916448615 +0000 UTC m=+276.733483232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-n64q8" (UID: "a300cc4c-bd94-4713-9d23-62561345e087") : secret "samples-operator-tls" not found Jul 21 01:58:07.463076 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:07.463030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" event={"ID":"468a2460-be01-417a-a822-e1664b4c121e","Type":"ContainerStarted","Data":"e5e32066c2ef350d0f5ed83c5f96b03535ffa52512a268d25515592bfb89b5c7"} Jul 21 01:58:07.480347 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:07.480297 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbswg" podStartSLOduration=1.04925519 podStartE2EDuration="2.480278248s" totalImagesPullingTime="1.431023058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:07.479416605 +0000 UTC m=+275.296451236" watchObservedRunningTime="2026-07-21 01:58:07.480278248 +0000 UTC m=+275.297312864" Jul 21 01:58:07.561491 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:07.561456 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:07.663197 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:07.663165 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:08.467592 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:08.467518 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" event={"ID":"84858abb-8a6a-4cc7-8766-f03b2b178381","Type":"ContainerStarted","Data":"3c60dbd9fd46fbfeef99190e83f7bda03c598d99d9ab176b7cb074a68de001c7"} Jul 21 01:58:08.469171 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:08.469142 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" event={"ID":"1822ea3d-9272-433d-9e6e-ad374d264b40","Type":"ContainerStarted","Data":"359da845645e3e312e4791a944b49b001809aaa16b5ca240a3955b36a159c8dd"} Jul 21 01:58:08.482072 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:08.482025 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" podStartSLOduration=0.912198828 podStartE2EDuration="3.482011457s" totalImagesPullingTime="2.569812629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:08.481151675 +0000 UTC m=+276.298186313" watchObservedRunningTime="2026-07-21 01:58:08.482011457 +0000 UTC m=+276.299046149" Jul 21 01:58:08.494734 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:08.494685 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" podStartSLOduration=0.904736637 podStartE2EDuration="3.49466893s" totalImagesPullingTime="2.589932293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:08.493712994 +0000 UTC m=+276.310747631" watchObservedRunningTime="2026-07-21 01:58:08.49466893 +0000 UTC m=+276.311703548" Jul 21 01:58:08.831762 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:08.831688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:08.831906 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:08.831828 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:08.831906 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:08.831886 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls podName:a48756e8-7067-4b95-92cf-e3b7cfde8a8a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:12.831868737 +0000 UTC m=+280.648903360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kgjhk" (UID: "a48756e8-7067-4b95-92cf-e3b7cfde8a8a") : secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:08.932701 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:08.932662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:08.932842 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:08.932767 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 01:58:08.932842 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:08.932816 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls podName:a300cc4c-bd94-4713-9d23-62561345e087 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:12.932804274 +0000 UTC m=+280.749838889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-n64q8" (UID: "a300cc4c-bd94-4713-9d23-62561345e087") : secret "samples-operator-tls" not found Jul 21 01:58:09.089052 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.089021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.099257 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.099232 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg"] Jul 21 01:58:09.102481 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.102458 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" Jul 21 01:58:09.105063 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.105043 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6s48j\"" Jul 21 01:58:09.105186 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.105164 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 21 01:58:09.105318 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.105170 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 21 01:58:09.110256 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.110238 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg"] Jul 21 01:58:09.182163 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.182136 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.234435 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.234398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqbnv\" (UniqueName: \"kubernetes.io/projected/dfb2ef4c-f952-4739-8ff2-75d58a62e67c-kube-api-access-tqbnv\") pod \"migrator-7cc9c7ffd7-cwlfg\" (UID: \"dfb2ef4c-f952-4739-8ff2-75d58a62e67c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" Jul 21 01:58:09.282604 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.282580 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.335502 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.335479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqbnv\" (UniqueName: \"kubernetes.io/projected/dfb2ef4c-f952-4739-8ff2-75d58a62e67c-kube-api-access-tqbnv\") pod \"migrator-7cc9c7ffd7-cwlfg\" (UID: \"dfb2ef4c-f952-4739-8ff2-75d58a62e67c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" Jul 21 01:58:09.342419 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.342354 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqbnv\" (UniqueName: \"kubernetes.io/projected/dfb2ef4c-f952-4739-8ff2-75d58a62e67c-kube-api-access-tqbnv\") pod \"migrator-7cc9c7ffd7-cwlfg\" (UID: \"dfb2ef4c-f952-4739-8ff2-75d58a62e67c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" Jul 21 01:58:09.412015 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.411981 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" Jul 21 01:58:09.508953 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.508924 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.528049 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.528019 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg"] Jul 21 01:58:09.530999 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:09.530970 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb2ef4c_f952_4739_8ff2_75d58a62e67c.slice/crio-3c976846874df4574ba3cd11c3c542e3532c079e0f4562193f2c4caf3120b10c WatchSource:0}: Error finding container 3c976846874df4574ba3cd11c3c542e3532c079e0f4562193f2c4caf3120b10c: Status 404 returned error can't find the container with id 3c976846874df4574ba3cd11c3c542e3532c079e0f4562193f2c4caf3120b10c Jul 21 01:58:09.609282 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.609206 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.793309 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.793282 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n25cz"] Jul 21 01:58:09.797389 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.797374 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.799623 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.799603 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 01:58:09.799878 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.799859 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pr29z\"" Jul 21 01:58:09.799953 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.799880 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 01:58:09.800185 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.800168 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 01:58:09.800237 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.800190 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 01:58:09.805684 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.805662 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n25cz"] Jul 21 01:58:09.838940 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.838917 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.839048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.839030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.839114 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.839069 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e74db41-069c-495d-b709-d950c6ce9c9b-crio-socket\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.839165 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.839110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5p7\" (UniqueName: \"kubernetes.io/projected/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-api-access-ns5p7\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.839218 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.839162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e74db41-069c-495d-b709-d950c6ce9c9b-data-volume\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.840023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.840001 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.893245 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.893198 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:09.939918 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.939893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.939990 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.939957 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.940035 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.939989 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e74db41-069c-495d-b709-d950c6ce9c9b-crio-socket\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.940035 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:09.940021 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 01:58:09.940114 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.940023 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5p7\" (UniqueName: \"kubernetes.io/projected/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-api-access-ns5p7\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.940114 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:09.940081 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:10.440065562 +0000 UTC m=+278.257100188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : configmap "kube-rbac-proxy" not found Jul 21 01:58:09.940114 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.940099 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e74db41-069c-495d-b709-d950c6ce9c9b-crio-socket\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.940257 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.940134 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e74db41-069c-495d-b709-d950c6ce9c9b-data-volume\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.940257 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:09.940170 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 01:58:09.940257 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:09.940210 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:10.440193299 +0000 UTC m=+278.257227920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : secret "insights-runtime-extractor-tls" not found Jul 21 01:58:09.940458 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.940440 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e74db41-069c-495d-b709-d950c6ce9c9b-data-volume\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.956899 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.956880 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5p7\" (UniqueName: \"kubernetes.io/projected/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-api-access-ns5p7\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:09.975357 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:09.975322 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:10.224102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:10.223961 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:10.321624 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:10.321591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:10.444249 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:10.444198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:10.444471 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:10.444367 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 01:58:10.444471 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:10.444459 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:11.444438688 +0000 UTC m=+279.261473307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : secret "insights-runtime-extractor-tls" not found Jul 21 01:58:10.444604 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:10.444495 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:10.444604 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:10.444597 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 01:58:10.444740 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:10.444639 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:11.44462671 +0000 UTC m=+279.261661329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : configmap "kube-rbac-proxy" not found Jul 21 01:58:10.477102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:10.477036 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" event={"ID":"dfb2ef4c-f952-4739-8ff2-75d58a62e67c","Type":"ContainerStarted","Data":"3c976846874df4574ba3cd11c3c542e3532c079e0f4562193f2c4caf3120b10c"} Jul 21 01:58:11.452670 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.452630 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:11.453105 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.452748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:11.453105 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:11.452784 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 01:58:11.453105 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:11.452840 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 01:58:11.453105 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:11.452850 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:13.452834614 +0000 UTC m=+281.269869231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : secret "insights-runtime-extractor-tls" not found Jul 21 01:58:11.453105 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:11.452892 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:13.452877299 +0000 UTC m=+281.269911933 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : configmap "kube-rbac-proxy" not found Jul 21 01:58:11.482318 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.482290 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" event={"ID":"dfb2ef4c-f952-4739-8ff2-75d58a62e67c","Type":"ContainerStarted","Data":"d1d55321bef7b5767ca8e42758eca6c52677d9d4b12cb9eda1b6fa26665c906d"} Jul 21 01:58:11.482481 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.482340 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" event={"ID":"dfb2ef4c-f952-4739-8ff2-75d58a62e67c","Type":"ContainerStarted","Data":"ea2e91315117a65930f6d2f605f006ca838586bb6565b5e7f355275831daa8e8"} Jul 21 01:58:11.495883 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.495833 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-cwlfg" podStartSLOduration=1.371783429 podStartE2EDuration="2.49581429s" totalImagesPullingTime="1.124030861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:11.495775042 +0000 UTC m=+279.312809680" watchObservedRunningTime="2026-07-21 01:58:11.49581429 +0000 UTC m=+279.312848923" Jul 21 01:58:11.569231 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.569202 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:11.669790 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.669755 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:11.854245 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.854221 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:11.907907 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:11.907881 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:12.588902 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:12.588878 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lf6nt_67d36da6-d97f-400d-bb0d-2d8986690d0c/dns-node-resolver/0.log" Jul 21 01:58:12.792623 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:12.792587 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:12.861139 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:12.861076 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:12.861246 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:12.861219 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:12.861283 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:12.861274 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls podName:a48756e8-7067-4b95-92cf-e3b7cfde8a8a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:20.861257646 +0000 UTC m=+288.678292263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kgjhk" (UID: "a48756e8-7067-4b95-92cf-e3b7cfde8a8a") : secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:12.962331 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:12.962307 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:12.962469 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:12.962457 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 01:58:12.962521 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:12.962511 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls podName:a300cc4c-bd94-4713-9d23-62561345e087 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:20.962497923 +0000 UTC m=+288.779532539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-n64q8" (UID: "a300cc4c-bd94-4713-9d23-62561345e087") : secret "samples-operator-tls" not found Jul 21 01:58:13.466381 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:13.466351 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:13.466527 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:13.466440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:13.466527 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:13.466496 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 01:58:13.466636 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:13.466537 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:17.466525489 +0000 UTC m=+285.283560105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : configmap "kube-rbac-proxy" not found Jul 21 01:58:13.466636 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:13.466497 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 01:58:13.466636 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:13.466615 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:17.466602885 +0000 UTC m=+285.283637501 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : secret "insights-runtime-extractor-tls" not found Jul 21 01:58:13.787728 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:13.787654 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d49sq_35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c/node-ca/0.log" Jul 21 01:58:14.588625 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:14.588594 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nbst9_e69a0874-21c0-4528-aec5-febde2de9500/serve-healthcheck-canary/0.log" Jul 21 01:58:14.788273 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:14.788242 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-cwlfg_dfb2ef4c-f952-4739-8ff2-75d58a62e67c/migrator/0.log" Jul 21 01:58:14.987688 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:14.987612 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-cwlfg_dfb2ef4c-f952-4739-8ff2-75d58a62e67c/graceful-termination/0.log" Jul 21 01:58:15.190130 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.190097 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-284bw_84858abb-8a6a-4cc7-8766-f03b2b178381/kube-storage-version-migrator-operator/0.log" Jul 21 01:58:15.194086 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:15.194060 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7hq8m" podUID="42ad014a-085f-4801-9869-6f03c196609a" Jul 21 01:58:15.194181 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:15.194060 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" podUID="e3e3d345-33d7-47a0-9128-3776ddef87bc" Jul 21 01:58:15.194181 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:15.194060 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" podUID="d5e054ae-c8cc-41cf-b425-41ed5e1206ff" Jul 21 01:58:15.271582 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.271521 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:15.470832 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.470791 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:15.493078 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.493053 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:58:15.493165 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.493074 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hq8m" Jul 21 01:58:15.493165 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.493069 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:58:15.571679 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.571615 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:15.679942 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.679915 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:15.776178 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.776150 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:15.873509 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.873472 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:15.975843 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:15.975812 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:16.775475 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:16.775438 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:17.374366 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:17.374329 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:17.494796 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:17.494767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:17.494907 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:17.494827 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:17.494907 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:17.494881 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 01:58:17.494973 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:17.494937 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:25.494922846 +0000 UTC m=+293.311957461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : configmap "kube-rbac-proxy" not found Jul 21 01:58:17.494973 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:17.494952 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 01:58:17.495047 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:17.494988 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls podName:3e74db41-069c-495d-b709-d950c6ce9c9b nodeName:}" failed. No retries permitted until 2026-07-21 01:58:25.494977805 +0000 UTC m=+293.312012421 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n25cz" (UID: "3e74db41-069c-495d-b709-d950c6ce9c9b") : secret "insights-runtime-extractor-tls" not found Jul 21 01:58:17.574757 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:17.574727 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:17.970142 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:17.970104 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:18.172468 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:18.172440 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:18.402611 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:18.402583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:18.402642 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") pod \"image-registry-6d4ff45c9b-65mlr\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:18.402679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402728 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402763 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402784 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert podName:d5e054ae-c8cc-41cf-b425-41ed5e1206ff nodeName:}" failed. No retries permitted until 2026-07-21 02:00:20.40277073 +0000 UTC m=+408.219805349 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-svqv9" (UID: "d5e054ae-c8cc-41cf-b425-41ed5e1206ff") : secret "networking-console-plugin-cert" not found Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402790 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402805 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d4ff45c9b-65mlr: secret "image-registry-tls" not found Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402797 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls podName:42ad014a-085f-4801-9869-6f03c196609a nodeName:}" failed. No retries permitted until 2026-07-21 02:00:20.402791025 +0000 UTC m=+408.219825640 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls") pod "dns-default-7hq8m" (UID: "42ad014a-085f-4801-9869-6f03c196609a") : secret "dns-default-metrics-tls" not found Jul 21 01:58:18.402988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:18.402846 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls podName:e3e3d345-33d7-47a0-9128-3776ddef87bc nodeName:}" failed. No retries permitted until 2026-07-21 02:00:20.402833839 +0000 UTC m=+408.219868456 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls") pod "image-registry-6d4ff45c9b-65mlr" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc") : secret "image-registry-tls" not found Jul 21 01:58:18.577563 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:18.577536 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:18.770234 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:18.770155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:19.089148 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:19.089120 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:19.190542 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:19.190517 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:19.475496 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:19.475413 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:19.778171 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:19.778086 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:20.175589 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:20.175560 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:20.375826 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:20.375784 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:20.576963 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:20.576862 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:20.773049 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:20.773018 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:20.921448 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:20.921421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:20.921593 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:20.921553 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:20.921643 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:20.921608 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls podName:a48756e8-7067-4b95-92cf-e3b7cfde8a8a nodeName:}" failed. No retries permitted until 2026-07-21 01:58:36.921594472 +0000 UTC m=+304.738629088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kgjhk" (UID: "a48756e8-7067-4b95-92cf-e3b7cfde8a8a") : secret "cluster-monitoring-operator-tls" not found Jul 21 01:58:21.022419 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.022385 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:21.024586 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.024569 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a300cc4c-bd94-4713-9d23-62561345e087-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-n64q8\" (UID: \"a300cc4c-bd94-4713-9d23-62561345e087\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:21.092820 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.092798 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" Jul 21 01:58:21.218065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.217985 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8"] Jul 21 01:58:21.511574 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.511509 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" event={"ID":"a300cc4c-bd94-4713-9d23-62561345e087","Type":"ContainerStarted","Data":"4e6ed8c25dc07e32d13720e8fa487d77754179255f424d78a5e068d13910013e"} Jul 21 01:58:21.570185 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.570152 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:21.810468 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.810373 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:21.970755 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:21.970706 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:22.268001 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:22.267966 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:22.372289 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:22.372259 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:22.575745 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:22.575655 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:22.973274 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:22.973246 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:23.519366 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:23.519331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" event={"ID":"a300cc4c-bd94-4713-9d23-62561345e087","Type":"ContainerStarted","Data":"d6c14ff9c89a1779430daa09e0cab3f41e673256171c6ba6f9f0ff350ce84b54"} Jul 21 01:58:23.519531 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:23.519394 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" event={"ID":"a300cc4c-bd94-4713-9d23-62561345e087","Type":"ContainerStarted","Data":"4f9118efda7a74953fac0daccff6325c1c74917521f43b8f9e4d6c680cbeebc8"} Jul 21 01:58:23.536245 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:23.536198 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-n64q8" podStartSLOduration=16.861967361 podStartE2EDuration="18.536181639s" totalImagesPullingTime="1.674214278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:23.535191762 +0000 UTC m=+291.352226401" watchObservedRunningTime="2026-07-21 01:58:23.536181639 +0000 UTC m=+291.353216275" Jul 21 01:58:23.572083 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:23.572060 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:23.976124 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:23.976094 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:24.379308 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:24.379273 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:24.577080 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:24.577046 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:24.973764 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:24.973737 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:25.558073 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.558051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:25.558432 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.558101 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:25.558578 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.558559 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e74db41-069c-495d-b709-d950c6ce9c9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:25.560303 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.560283 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e74db41-069c-495d-b709-d950c6ce9c9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n25cz\" (UID: \"3e74db41-069c-495d-b709-d950c6ce9c9b\") " pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:25.577925 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.577900 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:25.707011 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.706990 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n25cz" Jul 21 01:58:25.775561 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.775511 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:25.830314 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.830249 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n25cz"] Jul 21 01:58:25.833374 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:25.833350 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e74db41_069c_495d_b709_d950c6ce9c9b.slice/crio-036865c1746c59380d1b2d998513cfb2ba604b8d24124435f363502ada5de20d WatchSource:0}: Error finding container 036865c1746c59380d1b2d998513cfb2ba604b8d24124435f363502ada5de20d: Status 404 returned error can't find the container with id 036865c1746c59380d1b2d998513cfb2ba604b8d24124435f363502ada5de20d Jul 21 01:58:25.980329 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:25.980305 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:26.531260 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:26.531219 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n25cz" event={"ID":"3e74db41-069c-495d-b709-d950c6ce9c9b","Type":"ContainerStarted","Data":"036865c1746c59380d1b2d998513cfb2ba604b8d24124435f363502ada5de20d"} Jul 21 01:58:29.703321 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:29.703275 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-d6brs" podUID="956c97dd-8b8b-4dd0-b22b-43110503e7f7" Jul 21 01:58:31.553935 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:31.553894 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n25cz" event={"ID":"3e74db41-069c-495d-b709-d950c6ce9c9b","Type":"ContainerStarted","Data":"f4872f57130b26f92dae058c069cd5e9ce176d19536d7fc1b781d43943fe6d4c"} Jul 21 01:58:33.717114 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:33.717066 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:33.816188 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:33.816138 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:33.878220 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:33.878193 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 01:58:33.878368 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:33.878268 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 01:58:33.885023 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:33.885006 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 01:58:33.971180 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:33.971138 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 01:58:34.566422 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:34.566368 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n25cz" event={"ID":"3e74db41-069c-495d-b709-d950c6ce9c9b","Type":"ContainerStarted","Data":"dff091667468b0fc52341780234cb26b809602e56d9dd34dee00fedd7d625a47"} Jul 21 01:58:35.086130 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:35.086072 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:36.338209 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:36.338161 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:36.952609 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:36.952572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:36.955352 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:36.955329 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a48756e8-7067-4b95-92cf-e3b7cfde8a8a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kgjhk\" (UID: \"a48756e8-7067-4b95-92cf-e3b7cfde8a8a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:37.209560 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:37.209492 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ppntw\"" Jul 21 01:58:37.218208 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:37.218192 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" Jul 21 01:58:37.388857 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:37.388762 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk"] Jul 21 01:58:37.391562 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:37.391526 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48756e8_7067_4b95_92cf_e3b7cfde8a8a.slice/crio-2662dd9ff5134fea7d5fd2494f658c207f3dad2a0993dc528fbcf70fe5ac00a5 WatchSource:0}: Error finding container 2662dd9ff5134fea7d5fd2494f658c207f3dad2a0993dc528fbcf70fe5ac00a5: Status 404 returned error can't find the container with id 2662dd9ff5134fea7d5fd2494f658c207f3dad2a0993dc528fbcf70fe5ac00a5 Jul 21 01:58:37.577722 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:37.577636 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n25cz" event={"ID":"3e74db41-069c-495d-b709-d950c6ce9c9b","Type":"ContainerStarted","Data":"29d619cafc9f351f2b0292db151f71b06738cc663d2bcd0ce57a1e78c5426964"} Jul 21 01:58:37.578787 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:37.578762 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" event={"ID":"a48756e8-7067-4b95-92cf-e3b7cfde8a8a","Type":"ContainerStarted","Data":"2662dd9ff5134fea7d5fd2494f658c207f3dad2a0993dc528fbcf70fe5ac00a5"} Jul 21 01:58:37.594867 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:37.594827 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n25cz" podStartSLOduration=17.224015398 podStartE2EDuration="28.594812535s" totalImagesPullingTime="11.370797137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 01:58:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:37.593768196 +0000 UTC m=+305.410802835" watchObservedRunningTime="2026-07-21 01:58:37.594812535 +0000 UTC m=+305.411847173" Jul 21 01:58:39.434471 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.434369 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:39.534444 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.534391 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:39.587065 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.587034 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" event={"ID":"a48756e8-7067-4b95-92cf-e3b7cfde8a8a","Type":"ContainerStarted","Data":"892b5ef1bbe04eb859549a4718e62d1b632d45d852bbcfacff5cb92afcff5848"} Jul 21 01:58:39.591621 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.591598 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl"] Jul 21 01:58:39.594668 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.594645 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:39.596371 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.596346 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jbdx7\"" Jul 21 01:58:39.596560 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.596510 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 21 01:58:39.603797 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.603775 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl"] Jul 21 01:58:39.607138 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.607092 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kgjhk" podStartSLOduration=32.873544216 podStartE2EDuration="34.607074463s" totalImagesPullingTime="1.733530247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:39.605628152 +0000 UTC m=+307.422662790" watchObservedRunningTime="2026-07-21 01:58:39.607074463 +0000 UTC m=+307.424109101" Jul 21 01:58:39.612777 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.612747 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:39.776113 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.776048 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/472efe22-45c1-46e1-ab1a-2dcfac26ec46-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n2vl\" (UID: \"472efe22-45c1-46e1-ab1a-2dcfac26ec46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:39.876607 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.876577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/472efe22-45c1-46e1-ab1a-2dcfac26ec46-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n2vl\" (UID: \"472efe22-45c1-46e1-ab1a-2dcfac26ec46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:39.876722 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:39.876705 2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 21 01:58:39.876773 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:39.876763 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/472efe22-45c1-46e1-ab1a-2dcfac26ec46-tls-certificates podName:472efe22-45c1-46e1-ab1a-2dcfac26ec46 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:40.376749053 +0000 UTC m=+308.193783670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/472efe22-45c1-46e1-ab1a-2dcfac26ec46-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-8n2vl" (UID: "472efe22-45c1-46e1-ab1a-2dcfac26ec46") : secret "prometheus-operator-admission-webhook-tls" not found Jul 21 01:58:39.993541 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:39.993514 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:40.380236 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:40.380204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/472efe22-45c1-46e1-ab1a-2dcfac26ec46-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n2vl\" (UID: \"472efe22-45c1-46e1-ab1a-2dcfac26ec46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:40.382520 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:40.382496 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/472efe22-45c1-46e1-ab1a-2dcfac26ec46-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-8n2vl\" (UID: \"472efe22-45c1-46e1-ab1a-2dcfac26ec46\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:40.502940 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:40.502903 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:40.622438 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:40.622394 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl"] Jul 21 01:58:40.625246 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:40.625212 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472efe22_45c1_46e1_ab1a_2dcfac26ec46.slice/crio-d8aafb7d5f1bb153da8379ebbb37c8928255f842fa3f513b788cef89a48dd20a WatchSource:0}: Error finding container d8aafb7d5f1bb153da8379ebbb37c8928255f842fa3f513b788cef89a48dd20a: Status 404 returned error can't find the container with id d8aafb7d5f1bb153da8379ebbb37c8928255f842fa3f513b788cef89a48dd20a Jul 21 01:58:41.468196 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:41.468163 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:41.595898 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:41.595866 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" event={"ID":"472efe22-45c1-46e1-ab1a-2dcfac26ec46","Type":"ContainerStarted","Data":"d8aafb7d5f1bb153da8379ebbb37c8928255f842fa3f513b788cef89a48dd20a"} Jul 21 01:58:41.703049 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:41.703017 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:58:42.601078 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:42.601045 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" event={"ID":"472efe22-45c1-46e1-ab1a-2dcfac26ec46","Type":"ContainerStarted","Data":"6f00289de3f42ae44d0de8d3ed3ad51dfe3d23e68156980377405eec75f1a953"} Jul 21 01:58:42.601539 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:42.601243 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:42.605875 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:42.605853 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" Jul 21 01:58:42.616140 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:42.616100 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-8n2vl" podStartSLOduration=2.148709632 podStartE2EDuration="3.616087449s" totalImagesPullingTime="1.467377817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:42.615185151 +0000 UTC m=+310.432219790" watchObservedRunningTime="2026-07-21 01:58:42.616087449 +0000 UTC m=+310.433122086" Jul 21 01:58:43.624388 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.624326 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:43.663827 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.663798 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-728qf"] Jul 21 01:58:43.665883 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.665868 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.667854 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.667831 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 01:58:43.668304 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.668280 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 21 01:58:43.668389 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.668315 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 21 01:58:43.668492 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.668474 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hlrzq\"" Jul 21 01:58:43.676476 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.676459 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-728qf"] Jul 21 01:58:43.699339 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.699313 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:43.804114 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.804088 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.804209 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.804133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.804256 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.804209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.804291 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.804261 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmv76\" (UniqueName: \"kubernetes.io/projected/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-kube-api-access-dmv76\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.904636 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.904557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmv76\" (UniqueName: \"kubernetes.io/projected/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-kube-api-access-dmv76\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.904636 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.904622 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.904805 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.904659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.904805 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.904693 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.904805 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:43.904727 2561 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 21 01:58:43.904805 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:43.904800 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-tls podName:4e7f2101-c0ad-4340-8cc3-6c7171e801d1 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:44.404780761 +0000 UTC m=+312.221815390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-728qf" (UID: "4e7f2101-c0ad-4340-8cc3-6c7171e801d1") : secret "prometheus-operator-tls" not found Jul 21 01:58:43.905271 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.905254 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.907017 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.906999 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:43.915470 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:43.915452 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmv76\" (UniqueName: \"kubernetes.io/projected/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-kube-api-access-dmv76\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:44.340518 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:44.340486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:44.407198 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:44.407167 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:44.409361 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:44.409335 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e7f2101-c0ad-4340-8cc3-6c7171e801d1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-728qf\" (UID: \"4e7f2101-c0ad-4340-8cc3-6c7171e801d1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:44.575103 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:44.575079 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" Jul 21 01:58:44.692920 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:44.692888 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-728qf"] Jul 21 01:58:44.695418 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:44.695371 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7f2101_c0ad_4340_8cc3_6c7171e801d1.slice/crio-7afd9aa141a642dfd13372edbed0a2478b1988d55bf26d7d38bd7a003091d6ef WatchSource:0}: Error finding container 7afd9aa141a642dfd13372edbed0a2478b1988d55bf26d7d38bd7a003091d6ef: Status 404 returned error can't find the container with id 7afd9aa141a642dfd13372edbed0a2478b1988d55bf26d7d38bd7a003091d6ef Jul 21 01:58:45.611588 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:45.611553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" event={"ID":"4e7f2101-c0ad-4340-8cc3-6c7171e801d1","Type":"ContainerStarted","Data":"7afd9aa141a642dfd13372edbed0a2478b1988d55bf26d7d38bd7a003091d6ef"} Jul 21 01:58:46.029811 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:46.029774 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:46.617137 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:46.617093 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" event={"ID":"4e7f2101-c0ad-4340-8cc3-6c7171e801d1","Type":"ContainerStarted","Data":"6a464fdb9e789b19a49eb9d254edfd4285e512009be87115d21304e83d74da48"} Jul 21 01:58:46.617303 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:46.617151 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" event={"ID":"4e7f2101-c0ad-4340-8cc3-6c7171e801d1","Type":"ContainerStarted","Data":"12bd92bd838c8499ad0b9d697c5ac6a3276ae1a83dbf67f81ffea2004bc1eeed"} Jul 21 01:58:46.632971 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:46.632923 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-728qf" podStartSLOduration=2.539960668 podStartE2EDuration="3.632904211s" totalImagesPullingTime="1.092943543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:46.632646529 +0000 UTC m=+314.449681167" watchObservedRunningTime="2026-07-21 01:58:46.632904211 +0000 UTC m=+314.449938850" Jul 21 01:58:48.781297 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:48.781196 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:48.991081 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:48.991047 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.023021 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.022995 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s868j"] Jul 21 01:58:49.026570 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.026553 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.028588 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.028562 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t5p5s\"" Jul 21 01:58:49.028735 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.028656 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 01:58:49.028929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.028907 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 01:58:49.028929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.028923 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 01:58:49.083037 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.083010 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.145197 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-wtmp\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145298 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145210 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-sys\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145298 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145226 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr97n\" (UniqueName: \"kubernetes.io/projected/5da262fe-9cc2-40c9-966c-e78f980d7752-kube-api-access-mr97n\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145298 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145265 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-root\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145394 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145298 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-accelerators-collector-config\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145394 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145370 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-textfile\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145482 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145396 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-tls\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145482 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5da262fe-9cc2-40c9-966c-e78f980d7752-metrics-client-ca\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.145545 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.145490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.184343 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.184317 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.246595 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5da262fe-9cc2-40c9-966c-e78f980d7752-metrics-client-ca\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246696 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246696 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246621 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-wtmp\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246696 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246645 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-sys\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246696 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246663 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr97n\" (UniqueName: \"kubernetes.io/projected/5da262fe-9cc2-40c9-966c-e78f980d7752-kube-api-access-mr97n\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246853 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-root\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246853 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246727 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-sys\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246853 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-accelerators-collector-config\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246853 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246785 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-root\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.246853 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246788 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-wtmp\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.247030 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246858 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-textfile\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.247030 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.246883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-tls\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.247216 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.247195 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-textfile\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.247260 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.247202 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5da262fe-9cc2-40c9-966c-e78f980d7752-metrics-client-ca\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.247316 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.247280 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-accelerators-collector-config\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.248958 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.248940 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.249142 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.249124 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5da262fe-9cc2-40c9-966c-e78f980d7752-node-exporter-tls\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.254500 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.254473 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr97n\" (UniqueName: \"kubernetes.io/projected/5da262fe-9cc2-40c9-966c-e78f980d7752-kube-api-access-mr97n\") pod \"node-exporter-s868j\" (UID: \"5da262fe-9cc2-40c9-966c-e78f980d7752\") " pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.284624 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.284571 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.335737 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.335714 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s868j" Jul 21 01:58:49.348769 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:49.348748 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da262fe_9cc2_40c9_966c_e78f980d7752.slice/crio-d6399f263a9be04c782dd67a9455b141c2c98e634b3a64610cca46f76a062b87 WatchSource:0}: Error finding container d6399f263a9be04c782dd67a9455b141c2c98e634b3a64610cca46f76a062b87: Status 404 returned error can't find the container with id d6399f263a9be04c782dd67a9455b141c2c98e634b3a64610cca46f76a062b87 Jul 21 01:58:49.387733 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.387707 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.483789 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.483759 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.584972 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.584911 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:49.628029 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.628008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s868j" event={"ID":"5da262fe-9cc2-40c9-966c-e78f980d7752","Type":"ContainerStarted","Data":"d6399f263a9be04c782dd67a9455b141c2c98e634b3a64610cca46f76a062b87"} Jul 21 01:58:49.994326 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:49.994288 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:50.073005 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.072972 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:50.078191 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.078160 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 01:58:50.083198 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.083166 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.086578 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.086555 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 01:58:50.087487 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.087447 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 01:58:50.087620 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.087599 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 01:58:50.087687 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.087633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 01:58:50.087687 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.087633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 01:58:50.088040 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.088020 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 01:58:50.088828 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.088255 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-92b9v\"" Jul 21 01:58:50.088828 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.088261 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 01:58:50.088828 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.088317 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 01:58:50.088828 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.088557 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 01:58:50.096393 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.096373 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 01:58:50.156660 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156622 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhlb2\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-kube-api-access-dhlb2\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.156851 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-web-config\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.156851 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.156851 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.156851 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156828 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156934 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-config-out\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156961 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.156995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.157023 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157102 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.157068 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157337 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.157122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157337 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.157145 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.157337 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.157210 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.194944 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.194907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:50.258200 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-config-out\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258200 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258156 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258400 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258400 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258400 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258400 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258400 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258741 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258741 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhlb2\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-kube-api-access-dhlb2\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258741 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258681 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-web-config\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258741 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258733 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258961 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.258961 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.258793 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.259341 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.259100 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.259341 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.259127 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.260130 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.260103 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.261523 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.261481 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.261905 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.261869 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-config-out\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.262494 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.262454 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.263177 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.263138 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.263308 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.263275 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.263742 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.263701 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.263847 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.263755 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-web-config\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.264320 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.264276 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.265346 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.265319 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.267048 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.267029 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhlb2\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-kube-api-access-dhlb2\") pod \"alertmanager-main-0\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.394745 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.394723 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 01:58:50.534946 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.534890 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 01:58:50.542186 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:50.542161 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ae7178_9dbb_4892_b80c_49994c109d87.slice/crio-12e266270c81e593e2414b75d184168e98ca5b105b3890e5ef1d9c378edfb7e0 WatchSource:0}: Error finding container 12e266270c81e593e2414b75d184168e98ca5b105b3890e5ef1d9c378edfb7e0: Status 404 returned error can't find the container with id 12e266270c81e593e2414b75d184168e98ca5b105b3890e5ef1d9c378edfb7e0 Jul 21 01:58:50.633447 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.633377 2561 generic.go:356] "Generic (PLEG): container finished" podID="5da262fe-9cc2-40c9-966c-e78f980d7752" containerID="3212f8e1ff4d4b4853a020e363d4ca4f9e1fa9b8d84ad5710d86f012a7c80af5" exitCode=0 Jul 21 01:58:50.633563 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.633472 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s868j" event={"ID":"5da262fe-9cc2-40c9-966c-e78f980d7752","Type":"ContainerDied","Data":"3212f8e1ff4d4b4853a020e363d4ca4f9e1fa9b8d84ad5710d86f012a7c80af5"} Jul 21 01:58:50.634790 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:50.634765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"12e266270c81e593e2414b75d184168e98ca5b105b3890e5ef1d9c378edfb7e0"} Jul 21 01:58:51.646562 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:51.646531 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s868j" event={"ID":"5da262fe-9cc2-40c9-966c-e78f980d7752","Type":"ContainerStarted","Data":"8d39e97884f6579af604538a00e4e793a4fda09677c91ed1775cfbc48a71e736"} Jul 21 01:58:51.646869 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:51.646626 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s868j" event={"ID":"5da262fe-9cc2-40c9-966c-e78f980d7752","Type":"ContainerStarted","Data":"6bf367dc8e09769c775ff6a3d6adffd5a7ee89de597f2da55cd4fe91627b422a"} Jul 21 01:58:51.647919 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:51.647898 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4"} Jul 21 01:58:51.664344 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:51.664308 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s868j" podStartSLOduration=1.588697436 podStartE2EDuration="2.664295645s" totalImagesPullingTime="998.730974ms" totalInitContainerRuntime="76.867235ms" isStatefulPod=true podCreationTimestamp="2026-07-21 01:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:51.663846466 +0000 UTC m=+319.480881104" watchObservedRunningTime="2026-07-21 01:58:51.664295645 +0000 UTC m=+319.481330308" Jul 21 01:58:51.983718 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:51.983686 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:52.084546 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:52.084513 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:52.186497 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:52.186463 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:52.655253 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:52.655221 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4" exitCode=0 Jul 21 01:58:52.655713 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:52.655297 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4"} Jul 21 01:58:52.676606 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:52.676576 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:53.417970 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.417947 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79d4867d56-rh9sl"] Jul 21 01:58:53.421044 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.421028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.422843 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.422821 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 21 01:58:53.423521 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.423078 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9ks5v\"" Jul 21 01:58:53.423521 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.423227 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 21 01:58:53.423852 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.423824 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 21 01:58:53.424232 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.423978 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1tq9ee5973gcb\"" Jul 21 01:58:53.424232 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.424116 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 21 01:58:53.429617 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.429600 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79d4867d56-rh9sl"] Jul 21 01:58:53.489561 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489535 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9btcd\" (UniqueName: \"kubernetes.io/projected/faef3a87-2cac-4ca7-9249-ed6ad3553847-kube-api-access-9btcd\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.489674 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/faef3a87-2cac-4ca7-9249-ed6ad3553847-metrics-server-audit-profiles\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.489674 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faef3a87-2cac-4ca7-9249-ed6ad3553847-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.489754 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489691 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-secret-metrics-server-client-certs\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.489795 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489768 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-secret-metrics-server-tls\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.489830 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489793 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-client-ca-bundle\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.489869 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.489831 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/faef3a87-2cac-4ca7-9249-ed6ad3553847-audit-log\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590588 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/faef3a87-2cac-4ca7-9249-ed6ad3553847-audit-log\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590703 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9btcd\" (UniqueName: \"kubernetes.io/projected/faef3a87-2cac-4ca7-9249-ed6ad3553847-kube-api-access-9btcd\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590703 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/faef3a87-2cac-4ca7-9249-ed6ad3553847-metrics-server-audit-profiles\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590703 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faef3a87-2cac-4ca7-9249-ed6ad3553847-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590703 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-secret-metrics-server-client-certs\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590895 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-secret-metrics-server-tls\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.590895 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-client-ca-bundle\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.591003 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.590979 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/faef3a87-2cac-4ca7-9249-ed6ad3553847-audit-log\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.591498 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.591471 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faef3a87-2cac-4ca7-9249-ed6ad3553847-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.591666 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.591646 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/faef3a87-2cac-4ca7-9249-ed6ad3553847-metrics-server-audit-profiles\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.593047 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.593019 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-secret-metrics-server-client-certs\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.593128 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.593070 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-secret-metrics-server-tls\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.593128 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.593106 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/faef3a87-2cac-4ca7-9249-ed6ad3553847-client-ca-bundle\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.600132 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.600110 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9btcd\" (UniqueName: \"kubernetes.io/projected/faef3a87-2cac-4ca7-9249-ed6ad3553847-kube-api-access-9btcd\") pod \"metrics-server-79d4867d56-rh9sl\" (UID: \"faef3a87-2cac-4ca7-9249-ed6ad3553847\") " pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.662377 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.662352 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6"} Jul 21 01:58:53.662699 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.662461 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028"} Jul 21 01:58:53.662699 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.662491 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468"} Jul 21 01:58:53.662699 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.662516 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571"} Jul 21 01:58:53.662699 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.662539 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0"} Jul 21 01:58:53.731620 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.731565 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:58:53.785701 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.785674 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd"] Jul 21 01:58:53.790168 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.790140 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:53.792625 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.792599 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xcmj6\"" Jul 21 01:58:53.792843 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.792826 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 21 01:58:53.794370 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.794343 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd"] Jul 21 01:58:53.800742 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.800456 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:53.866018 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.865997 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79d4867d56-rh9sl"] Jul 21 01:58:53.868251 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:53.868226 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaef3a87_2cac_4ca7_9249_ed6ad3553847.slice/crio-9b2fbce54d1cc5b1ed2c6fbf1aa64a6687116e899123ff621eb73edb3c9812ba WatchSource:0}: Error finding container 9b2fbce54d1cc5b1ed2c6fbf1aa64a6687116e899123ff621eb73edb3c9812ba: Status 404 returned error can't find the container with id 9b2fbce54d1cc5b1ed2c6fbf1aa64a6687116e899123ff621eb73edb3c9812ba Jul 21 01:58:53.893682 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.893660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-967kt\" (UniqueName: \"kubernetes.io/projected/97553225-4513-49a4-82c7-dc676e038e86-kube-api-access-967kt\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:53.893770 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.893704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97553225-4513-49a4-82c7-dc676e038e86-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:53.994852 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.994784 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-967kt\" (UniqueName: \"kubernetes.io/projected/97553225-4513-49a4-82c7-dc676e038e86-kube-api-access-967kt\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:53.994852 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:53.994836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97553225-4513-49a4-82c7-dc676e038e86-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:53.994988 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:53.994937 2561 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 21 01:58:53.995039 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:53.994993 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97553225-4513-49a4-82c7-dc676e038e86-monitoring-plugin-cert podName:97553225-4513-49a4-82c7-dc676e038e86 nodeName:}" failed. No retries permitted until 2026-07-21 01:58:54.494976013 +0000 UTC m=+322.312010630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/97553225-4513-49a4-82c7-dc676e038e86-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-cjwhd" (UID: "97553225-4513-49a4-82c7-dc676e038e86") : secret "monitoring-plugin-cert" not found Jul 21 01:58:54.003616 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.003592 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-967kt\" (UniqueName: \"kubernetes.io/projected/97553225-4513-49a4-82c7-dc676e038e86-kube-api-access-967kt\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:54.246232 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.246158 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:54.499836 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.499768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97553225-4513-49a4-82c7-dc676e038e86-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:54.502345 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.502323 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97553225-4513-49a4-82c7-dc676e038e86-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjwhd\" (UID: \"97553225-4513-49a4-82c7-dc676e038e86\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:54.670014 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.669918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerStarted","Data":"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4"} Jul 21 01:58:54.671344 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.671313 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" event={"ID":"faef3a87-2cac-4ca7-9249-ed6ad3553847","Type":"ContainerStarted","Data":"9b2fbce54d1cc5b1ed2c6fbf1aa64a6687116e899123ff621eb73edb3c9812ba"} Jul 21 01:58:54.693808 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.693729 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.284692604 podStartE2EDuration="4.693712074s" totalImagesPullingTime="3.394415048s" totalInitContainerRuntime="14.604422ms" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:54.692966291 +0000 UTC m=+322.510000928" watchObservedRunningTime="2026-07-21 01:58:54.693712074 +0000 UTC m=+322.510746713" Jul 21 01:58:54.704474 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.704451 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:54.850155 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:54.850126 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd"] Jul 21 01:58:54.853569 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:54.853544 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97553225_4513_49a4_82c7_dc676e038e86.slice/crio-a236932c1143b719bc872aa7d5e6147c69ef6876af2eeaa7b8577020ee39a6f4 WatchSource:0}: Error finding container a236932c1143b719bc872aa7d5e6147c69ef6876af2eeaa7b8577020ee39a6f4: Status 404 returned error can't find the container with id a236932c1143b719bc872aa7d5e6147c69ef6876af2eeaa7b8577020ee39a6f4 Jul 21 01:58:55.309790 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.309744 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 01:58:55.313805 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.313776 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.320482 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320457 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 21 01:58:55.320594 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320462 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 21 01:58:55.320594 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320504 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 21 01:58:55.320704 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320626 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ctz2d\"" Jul 21 01:58:55.320918 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320901 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 21 01:58:55.320918 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320910 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 21 01:58:55.321064 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320960 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 21 01:58:55.321064 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.320973 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 21 01:58:55.321312 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.321298 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 21 01:58:55.321312 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.321304 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 21 01:58:55.321487 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.321363 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 21 01:58:55.321487 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.321368 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 21 01:58:55.321595 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.321562 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 21 01:58:55.321646 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.321602 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6m4bdd03ghitu\"" Jul 21 01:58:55.324098 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.323188 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 21 01:58:55.324098 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.324076 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:55.324996 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.324830 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 21 01:58:55.353030 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.353008 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 01:58:55.407542 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.407513 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-config\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.407748 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.407730 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-config-out\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.407849 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.407835 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.407968 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.407953 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408180 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408266 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408754 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408858 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408751 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-web-config\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408858 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhzv2\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-kube-api-access-nhzv2\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408858 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408797 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.408858 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408820 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409009 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409009 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408913 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409009 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408937 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409009 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.408989 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409147 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.409032 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409147 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.409057 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409147 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.409078 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409147 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.409115 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.409147 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.409137 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510067 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510029 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-config\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510224 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510095 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-config-out\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510224 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510224 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510157 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510224 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510224 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510207 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510245 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-web-config\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhzv2\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-kube-api-access-nhzv2\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510315 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510398 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510437 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510506 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510957 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510957 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510562 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510957 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510957 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510630 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.510957 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.510654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.511860 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.511788 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.512567 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.512542 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.512710 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.512620 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.512924 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.512897 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-config-out\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.513217 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.513196 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.513333 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.513206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-config\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.514540 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.513713 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.514540 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.514076 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.514540 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.514495 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.515381 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.515326 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.516118 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.516089 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.516973 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.516683 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.516973 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.516845 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.517665 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.517303 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.517665 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.517337 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.517665 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.517624 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.517856 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.517787 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.517931 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.517899 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.518332 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.518310 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-web-config\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.519920 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.519855 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhzv2\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-kube-api-access-nhzv2\") pod \"prometheus-k8s-0\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.629149 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.629122 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:58:55.679902 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.679698 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" event={"ID":"faef3a87-2cac-4ca7-9249-ed6ad3553847","Type":"ContainerStarted","Data":"b248ba2b68e2cc8de8110b9063508a042a5bd2cb96ca86475f0236156306b7c0"} Jul 21 01:58:55.685128 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.685096 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" event={"ID":"97553225-4513-49a4-82c7-dc676e038e86","Type":"ContainerStarted","Data":"a236932c1143b719bc872aa7d5e6147c69ef6876af2eeaa7b8577020ee39a6f4"} Jul 21 01:58:55.698110 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.698055 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" podStartSLOduration=1.2369438640000001 podStartE2EDuration="2.698035524s" totalImagesPullingTime="1.46109166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:55.696865359 +0000 UTC m=+323.513899999" watchObservedRunningTime="2026-07-21 01:58:55.698035524 +0000 UTC m=+323.515070161" Jul 21 01:58:55.763892 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.763854 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d4ff45c9b-65mlr"] Jul 21 01:58:55.766248 ip-10-0-135-168 kubenswrapper[2561]: E0721 01:58:55.764317 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" podUID="e3e3d345-33d7-47a0-9128-3776ddef87bc" Jul 21 01:58:55.831301 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.831234 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 01:58:55.834393 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:58:55.834361 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25adda6b_afb9_4e1f_81a0_ff7504f59004.slice/crio-1023c4beae046135aeb49ca602487cc6be029f92c8d2e2ba9636d3a15620c597 WatchSource:0}: Error finding container 1023c4beae046135aeb49ca602487cc6be029f92c8d2e2ba9636d3a15620c597: Status 404 returned error can't find the container with id 1023c4beae046135aeb49ca602487cc6be029f92c8d2e2ba9636d3a15620c597 Jul 21 01:58:55.845051 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.845021 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:55.977745 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:55.977711 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:56.077990 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.077942 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:58:56.690624 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.690544 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91" exitCode=0 Jul 21 01:58:56.690999 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.690624 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91"} Jul 21 01:58:56.690999 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.690707 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"1023c4beae046135aeb49ca602487cc6be029f92c8d2e2ba9636d3a15620c597"} Jul 21 01:58:56.692143 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.692116 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" event={"ID":"97553225-4513-49a4-82c7-dc676e038e86","Type":"ContainerStarted","Data":"8f18ee908e5f5365eb21deb05d38286d250ab114228035dd108f1e2a880faaed"} Jul 21 01:58:56.692257 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.692241 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:58:56.692546 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.692524 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:56.697385 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.697368 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" Jul 21 01:58:56.698940 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.698924 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:58:56.721929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.721899 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-certificates\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.721929 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.721935 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-installation-pull-secrets\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.722083 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.721964 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3e3d345-33d7-47a0-9128-3776ddef87bc-ca-trust-extracted\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.722083 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.721981 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q878\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-kube-api-access-9q878\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.722083 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.721999 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-image-registry-private-configuration\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.722083 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.722059 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-trusted-ca\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.722320 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.722104 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-bound-sa-token\") pod \"e3e3d345-33d7-47a0-9128-3776ddef87bc\" (UID: \"e3e3d345-33d7-47a0-9128-3776ddef87bc\") " Jul 21 01:58:56.722320 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.722225 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3e3d345-33d7-47a0-9128-3776ddef87bc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 01:58:56.722320 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.722260 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 01:58:56.722775 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.722567 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 01:58:56.723312 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.723289 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-certificates\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:56.723452 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.723436 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e3e3d345-33d7-47a0-9128-3776ddef87bc-ca-trust-extracted\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:56.723560 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.723547 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3e3d345-33d7-47a0-9128-3776ddef87bc-trusted-ca\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:56.724851 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.724827 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-kube-api-access-9q878" (OuterVolumeSpecName: "kube-api-access-9q878") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "kube-api-access-9q878". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 01:58:56.724945 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.724900 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 01:58:56.724945 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.724933 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 01:58:56.725542 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.725520 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e3e3d345-33d7-47a0-9128-3776ddef87bc" (UID: "e3e3d345-33d7-47a0-9128-3776ddef87bc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 01:58:56.745595 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.745555 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjwhd" podStartSLOduration=2.233207729 podStartE2EDuration="3.745541942s" totalImagesPullingTime="1.512334213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:58:56.745112922 +0000 UTC m=+324.562147560" watchObservedRunningTime="2026-07-21 01:58:56.745541942 +0000 UTC m=+324.562576592" Jul 21 01:58:56.824737 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.824709 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-bound-sa-token\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:56.824737 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.824732 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-installation-pull-secrets\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:56.824737 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.824741 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q878\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-kube-api-access-9q878\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:56.824912 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:56.824752 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e3e3d345-33d7-47a0-9128-3776ddef87bc-image-registry-private-configuration\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:57.700525 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:57.698237 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4ff45c9b-65mlr" Jul 21 01:58:57.732347 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:57.732311 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d4ff45c9b-65mlr"] Jul 21 01:58:57.735313 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:57.735288 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d4ff45c9b-65mlr"] Jul 21 01:58:57.833317 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:57.833289 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e3e3d345-33d7-47a0-9128-3776ddef87bc-registry-tls\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 01:58:58.709358 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:58:58.709324 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3e3d345-33d7-47a0-9128-3776ddef87bc" path="/var/lib/kubelet/pods/e3e3d345-33d7-47a0-9128-3776ddef87bc/volumes" Jul 21 01:59:00.047766 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:00.047720 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:00.148545 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:00.148510 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:00.711196 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:00.711161 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e"} Jul 21 01:59:00.711352 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:00.711221 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37"} Jul 21 01:59:01.005397 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.005289 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.101550 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.101506 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.201267 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.201179 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.301052 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.300944 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.470333 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.470241 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.658492 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.658461 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.713459 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.713428 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.890614 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.890582 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:01.943877 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:01.943795 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:02.721684 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:02.721653 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546"} Jul 21 01:59:02.722100 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:02.721726 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4"} Jul 21 01:59:02.722100 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:02.721757 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455"} Jul 21 01:59:02.722100 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:02.721786 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerStarted","Data":"4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a"} Jul 21 01:59:02.747605 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:02.747548 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.470723957 podStartE2EDuration="7.747529839s" totalImagesPullingTime="5.247817303s" totalInitContainerRuntime="28.988579ms" isStatefulPod=false podCreationTimestamp="2026-07-21 01:58:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:59:02.746607889 +0000 UTC m=+330.563642526" watchObservedRunningTime="2026-07-21 01:59:02.747529839 +0000 UTC m=+330.564564478" Jul 21 01:59:02.871962 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:02.871930 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:03.063599 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:03.063522 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:03.227852 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:03.227814 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:04.081107 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:04.081072 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:04.134522 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:04.134494 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:05.629278 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:05.629251 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:59:06.322142 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:06.322108 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:06.422441 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:06.422390 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:08.107816 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:08.107783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:08.161430 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:08.161389 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:09.063975 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:09.063939 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:13.732621 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:13.732590 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:59:13.732989 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:13.732639 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:59:14.905574 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:14.905526 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:15.390084 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:15.390049 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:15.491777 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:15.491740 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:16.301884 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:16.301846 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:16.410171 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:16.410145 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:18.787307 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:18.787238 2561 generic.go:356] "Generic (PLEG): container finished" podID="1822ea3d-9272-433d-9e6e-ad374d264b40" containerID="359da845645e3e312e4791a944b49b001809aaa16b5ca240a3955b36a159c8dd" exitCode=0 Jul 21 01:59:18.787307 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:18.787286 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" event={"ID":"1822ea3d-9272-433d-9e6e-ad374d264b40","Type":"ContainerDied","Data":"359da845645e3e312e4791a944b49b001809aaa16b5ca240a3955b36a159c8dd"} Jul 21 01:59:18.787698 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:18.787601 2561 scope.go:117] "RemoveContainer" containerID="359da845645e3e312e4791a944b49b001809aaa16b5ca240a3955b36a159c8dd" Jul 21 01:59:19.792188 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:19.792150 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wzzz" event={"ID":"1822ea3d-9272-433d-9e6e-ad374d264b40","Type":"ContainerStarted","Data":"1e9f1c0b1194c405f2580e4328bebbd3901580369ed3aa394ad41de0d37ce5ea"} Jul 21 01:59:20.089608 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:20.089578 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:20.146871 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:20.146841 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:20.206340 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:20.206310 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:27.691325 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:27.691244 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:27.741936 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:27.741908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:28.698760 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:28.698727 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:28.749290 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:28.749256 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:30.490696 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:30.490655 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:30.698746 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:30.698713 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:30.751510 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:30.751431 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:30.902420 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:30.902362 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:30.973544 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:30.973509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:33.737563 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:33.737537 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:59:33.741314 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:33.741291 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79d4867d56-rh9sl" Jul 21 01:59:33.842252 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:33.842220 2561 generic.go:356] "Generic (PLEG): container finished" podID="84858abb-8a6a-4cc7-8766-f03b2b178381" containerID="3c60dbd9fd46fbfeef99190e83f7bda03c598d99d9ab176b7cb074a68de001c7" exitCode=0 Jul 21 01:59:33.842398 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:33.842294 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" event={"ID":"84858abb-8a6a-4cc7-8766-f03b2b178381","Type":"ContainerDied","Data":"3c60dbd9fd46fbfeef99190e83f7bda03c598d99d9ab176b7cb074a68de001c7"} Jul 21 01:59:33.842628 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:33.842612 2561 scope.go:117] "RemoveContainer" containerID="3c60dbd9fd46fbfeef99190e83f7bda03c598d99d9ab176b7cb074a68de001c7" Jul 21 01:59:34.848630 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:34.848600 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-284bw" event={"ID":"84858abb-8a6a-4cc7-8766-f03b2b178381","Type":"ContainerStarted","Data":"f99f949355748301856fc9e9235b0e645168d2d9a303451821390e66eae1bf3e"} Jul 21 01:59:35.090281 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:35.090244 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:35.161604 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:35.161542 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:40.747423 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:40.747368 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nbst9_e69a0874-21c0-4528-aec5-febde2de9500/serve-healthcheck-canary/0.log" Jul 21 01:59:42.430769 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:42.430729 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:42.531024 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:42.530991 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:42.584832 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:42.584798 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:46.524490 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:46.524459 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:59:46.526668 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:46.526640 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/956c97dd-8b8b-4dd0-b22b-43110503e7f7-metrics-certs\") pod \"network-metrics-daemon-d6brs\" (UID: \"956c97dd-8b8b-4dd0-b22b-43110503e7f7\") " pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:59:46.805805 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:46.805740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8gmrt\"" Jul 21 01:59:46.814328 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:46.814310 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6brs" Jul 21 01:59:46.938005 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:46.937983 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d6brs"] Jul 21 01:59:46.940756 ip-10-0-135-168 kubenswrapper[2561]: W0721 01:59:46.940731 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956c97dd_8b8b_4dd0_b22b_43110503e7f7.slice/crio-88bcbc37040adb0b726983454edeef9d5e3837aad981cddcaaf007cbffadd16a WatchSource:0}: Error finding container 88bcbc37040adb0b726983454edeef9d5e3837aad981cddcaaf007cbffadd16a: Status 404 returned error can't find the container with id 88bcbc37040adb0b726983454edeef9d5e3837aad981cddcaaf007cbffadd16a Jul 21 01:59:47.914860 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:47.914824 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6brs" event={"ID":"956c97dd-8b8b-4dd0-b22b-43110503e7f7","Type":"ContainerStarted","Data":"88bcbc37040adb0b726983454edeef9d5e3837aad981cddcaaf007cbffadd16a"} Jul 21 01:59:48.920036 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:48.920004 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6brs" event={"ID":"956c97dd-8b8b-4dd0-b22b-43110503e7f7","Type":"ContainerStarted","Data":"33b90fccc49a95c455c765ba93d511abe155da80b2ee6195aabe788966d2ad11"} Jul 21 01:59:48.920485 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:48.920054 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6brs" event={"ID":"956c97dd-8b8b-4dd0-b22b-43110503e7f7","Type":"ContainerStarted","Data":"2c4684d960279164bcfc7b38437cbeaef16c593504ba101a9afa761b44346b8f"} Jul 21 01:59:48.935732 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:48.935686 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d6brs" podStartSLOduration=374.713842711 podStartE2EDuration="6m15.935672386s" totalImagesPullingTime="1.221829675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 01:59:48.934124291 +0000 UTC m=+376.751158929" watchObservedRunningTime="2026-07-21 01:59:48.935672386 +0000 UTC m=+376.752707024" Jul 21 01:59:49.043876 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:49.043847 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:49.102641 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:49.102613 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:49.155192 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:49.155155 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:50.087343 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:50.087302 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:50.140564 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:50.140531 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 01:59:55.629781 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:55.629747 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:59:55.648823 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:55.648799 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 01:59:55.964363 ip-10-0-135-168 kubenswrapper[2561]: I0721 01:59:55.964288 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:08.260473 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:08.260435 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:08.466710 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:08.466652 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:08.561797 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:08.561712 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:09.334843 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.334811 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 02:00:09.335237 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.335147 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="alertmanager" containerID="cri-o://19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0" gracePeriod=120 Jul 21 02:00:09.335307 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.335239 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="prom-label-proxy" containerID="cri-o://e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4" gracePeriod=120 Jul 21 02:00:09.335307 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.335254 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-web" containerID="cri-o://55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468" gracePeriod=120 Jul 21 02:00:09.335307 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.335279 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="config-reloader" containerID="cri-o://4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571" gracePeriod=120 Jul 21 02:00:09.335491 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.335334 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy" containerID="cri-o://1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028" gracePeriod=120 Jul 21 02:00:09.335491 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:09.335218 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-metric" containerID="cri-o://a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6" gracePeriod=120 Jul 21 02:00:10.013613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013581 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4" exitCode=0 Jul 21 02:00:10.013613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013605 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028" exitCode=0 Jul 21 02:00:10.013613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013613 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571" exitCode=0 Jul 21 02:00:10.013613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013619 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0" exitCode=0 Jul 21 02:00:10.013865 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013652 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4"} Jul 21 02:00:10.013865 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013713 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028"} Jul 21 02:00:10.013865 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013740 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571"} Jul 21 02:00:10.013865 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.013763 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0"} Jul 21 02:00:10.576766 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.576746 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:10.739639 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739563 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739639 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739597 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-main-tls\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739639 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739613 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-cluster-tls-config\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739639 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739639 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhlb2\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-kube-api-access-dhlb2\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739684 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-config-volume\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739725 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739845 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739897 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-trusted-ca-bundle\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739940 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-main-db\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.739973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.739970 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-config-out\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.740268 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.740033 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-tls-assets\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.740268 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.740067 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-web-config\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.740268 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.740116 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-metrics-client-ca\") pod \"51ae7178-9dbb-4892-b80c-49994c109d87\" (UID: \"51ae7178-9dbb-4892-b80c-49994c109d87\") " Jul 21 02:00:10.740929 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.740753 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:10.741049 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.740939 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:00:10.741801 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.741758 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:10.742592 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.742564 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.743048 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.743022 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.743137 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.743029 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-config-volume" (OuterVolumeSpecName: "config-volume") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.743420 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.743377 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.743830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.743796 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-kube-api-access-dhlb2" (OuterVolumeSpecName: "kube-api-access-dhlb2") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "kube-api-access-dhlb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 02:00:10.743924 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.743839 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-config-out" (OuterVolumeSpecName: "config-out") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:00:10.743924 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.743859 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 02:00:10.744031 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.744008 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.746762 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.746741 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.754086 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.754063 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-web-config" (OuterVolumeSpecName: "web-config") pod "51ae7178-9dbb-4892-b80c-49994c109d87" (UID: "51ae7178-9dbb-4892-b80c-49994c109d87"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:10.841611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841588 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841611 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841621 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841632 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-alertmanager-main-db\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841641 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51ae7178-9dbb-4892-b80c-49994c109d87-config-out\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841649 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-tls-assets\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841657 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-web-config\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841664 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51ae7178-9dbb-4892-b80c-49994c109d87-metrics-client-ca\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841672 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841681 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-secret-alertmanager-main-tls\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841688 2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-cluster-tls-config\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841697 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhlb2\" (UniqueName: \"kubernetes.io/projected/51ae7178-9dbb-4892-b80c-49994c109d87-kube-api-access-dhlb2\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:10.841731 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:10.841704 2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51ae7178-9dbb-4892-b80c-49994c109d87-config-volume\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:11.021456 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021370 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6" exitCode=0 Jul 21 02:00:11.021456 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021400 2561 generic.go:356] "Generic (PLEG): container finished" podID="51ae7178-9dbb-4892-b80c-49994c109d87" containerID="55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468" exitCode=0 Jul 21 02:00:11.021605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021456 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6"} Jul 21 02:00:11.021605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021496 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.021605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468"} Jul 21 02:00:11.021605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021560 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51ae7178-9dbb-4892-b80c-49994c109d87","Type":"ContainerDied","Data":"12e266270c81e593e2414b75d184168e98ca5b105b3890e5ef1d9c378edfb7e0"} Jul 21 02:00:11.021605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.021591 2561 scope.go:117] "RemoveContainer" containerID="e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4" Jul 21 02:00:11.030258 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.030240 2561 scope.go:117] "RemoveContainer" containerID="a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6" Jul 21 02:00:11.037060 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.037043 2561 scope.go:117] "RemoveContainer" containerID="1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028" Jul 21 02:00:11.043527 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.043508 2561 scope.go:117] "RemoveContainer" containerID="55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468" Jul 21 02:00:11.045444 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.045424 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 02:00:11.049031 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.049007 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 02:00:11.051274 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.051259 2561 scope.go:117] "RemoveContainer" containerID="4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571" Jul 21 02:00:11.057908 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.057892 2561 scope.go:117] "RemoveContainer" containerID="19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0" Jul 21 02:00:11.064546 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.064532 2561 scope.go:117] "RemoveContainer" containerID="a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4" Jul 21 02:00:11.071222 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.071207 2561 scope.go:117] "RemoveContainer" containerID="e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4" Jul 21 02:00:11.071504 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.071485 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4\": container with ID starting with e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4 not found: ID does not exist" containerID="e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4" Jul 21 02:00:11.071552 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.071513 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4"} err="failed to get container status \"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4\": rpc error: code = NotFound desc = could not find container \"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4\": container with ID starting with e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4 not found: ID does not exist" Jul 21 02:00:11.071552 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.071540 2561 scope.go:117] "RemoveContainer" containerID="a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6" Jul 21 02:00:11.071790 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.071772 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6\": container with ID starting with a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6 not found: ID does not exist" containerID="a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6" Jul 21 02:00:11.071830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.071796 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6"} err="failed to get container status \"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6\": rpc error: code = NotFound desc = could not find container \"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6\": container with ID starting with a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6 not found: ID does not exist" Jul 21 02:00:11.071830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.071811 2561 scope.go:117] "RemoveContainer" containerID="1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028" Jul 21 02:00:11.072040 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.072024 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028\": container with ID starting with 1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028 not found: ID does not exist" containerID="1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028" Jul 21 02:00:11.072079 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072047 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028"} err="failed to get container status \"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028\": rpc error: code = NotFound desc = could not find container \"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028\": container with ID starting with 1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028 not found: ID does not exist" Jul 21 02:00:11.072079 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072063 2561 scope.go:117] "RemoveContainer" containerID="55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468" Jul 21 02:00:11.072276 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.072262 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468\": container with ID starting with 55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468 not found: ID does not exist" containerID="55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468" Jul 21 02:00:11.072323 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072278 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468"} err="failed to get container status \"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468\": rpc error: code = NotFound desc = could not find container \"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468\": container with ID starting with 55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468 not found: ID does not exist" Jul 21 02:00:11.072323 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072290 2561 scope.go:117] "RemoveContainer" containerID="4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571" Jul 21 02:00:11.072568 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.072549 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571\": container with ID starting with 4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571 not found: ID does not exist" containerID="4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571" Jul 21 02:00:11.072613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072573 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571"} err="failed to get container status \"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571\": rpc error: code = NotFound desc = could not find container \"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571\": container with ID starting with 4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571 not found: ID does not exist" Jul 21 02:00:11.072613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072586 2561 scope.go:117] "RemoveContainer" containerID="19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0" Jul 21 02:00:11.072820 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.072803 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0\": container with ID starting with 19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0 not found: ID does not exist" containerID="19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0" Jul 21 02:00:11.072882 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072837 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0"} err="failed to get container status \"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0\": rpc error: code = NotFound desc = could not find container \"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0\": container with ID starting with 19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0 not found: ID does not exist" Jul 21 02:00:11.072882 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.072859 2561 scope.go:117] "RemoveContainer" containerID="a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4" Jul 21 02:00:11.073090 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:11.073074 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4\": container with ID starting with a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4 not found: ID does not exist" containerID="a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4" Jul 21 02:00:11.073127 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073094 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4"} err="failed to get container status \"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4\": rpc error: code = NotFound desc = could not find container \"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4\": container with ID starting with a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4 not found: ID does not exist" Jul 21 02:00:11.073127 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073107 2561 scope.go:117] "RemoveContainer" containerID="e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4" Jul 21 02:00:11.073307 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073284 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4"} err="failed to get container status \"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4\": rpc error: code = NotFound desc = could not find container \"e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4\": container with ID starting with e8fa30a7ac2ec2ad1014049474343fc715ac5ad8459811497fcb1c0ad488cda4 not found: ID does not exist" Jul 21 02:00:11.073350 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073309 2561 scope.go:117] "RemoveContainer" containerID="a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6" Jul 21 02:00:11.073526 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073505 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6"} err="failed to get container status \"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6\": rpc error: code = NotFound desc = could not find container \"a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6\": container with ID starting with a23402f1975e4b0dfc52bcb8c5d053180be37bec350fed736d7a66546be8ddd6 not found: ID does not exist" Jul 21 02:00:11.073526 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073524 2561 scope.go:117] "RemoveContainer" containerID="1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028" Jul 21 02:00:11.073727 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073710 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028"} err="failed to get container status \"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028\": rpc error: code = NotFound desc = could not find container \"1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028\": container with ID starting with 1541207aee49889daa0af0d9ffc868d66e5a501708d0639c626a85c7caefc028 not found: ID does not exist" Jul 21 02:00:11.073799 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073728 2561 scope.go:117] "RemoveContainer" containerID="55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468" Jul 21 02:00:11.073950 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073933 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468"} err="failed to get container status \"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468\": rpc error: code = NotFound desc = could not find container \"55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468\": container with ID starting with 55c0a6496feea8f39d2c57fc0763906ab8affffb4d66f711f076f13988794468 not found: ID does not exist" Jul 21 02:00:11.073996 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.073950 2561 scope.go:117] "RemoveContainer" containerID="4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571" Jul 21 02:00:11.074155 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.074127 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571"} err="failed to get container status \"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571\": rpc error: code = NotFound desc = could not find container \"4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571\": container with ID starting with 4f688c69033aeca565d6eab007f650d09280a3ee52cd176f5c6592b72b952571 not found: ID does not exist" Jul 21 02:00:11.074196 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.074154 2561 scope.go:117] "RemoveContainer" containerID="19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0" Jul 21 02:00:11.074513 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.074399 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0"} err="failed to get container status \"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0\": rpc error: code = NotFound desc = could not find container \"19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0\": container with ID starting with 19d0774004e0213f2d31347e6ec42f046ce499f465b08d42f1b75ba3ad7caae0 not found: ID does not exist" Jul 21 02:00:11.074638 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.074516 2561 scope.go:117] "RemoveContainer" containerID="a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4" Jul 21 02:00:11.074861 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.074828 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4"} err="failed to get container status \"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4\": rpc error: code = NotFound desc = could not find container \"a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4\": container with ID starting with a4ef753ced544e434a638a0aa3fbdd0681f73def6665e72bf3a1d736560980e4 not found: ID does not exist" Jul 21 02:00:11.076225 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076208 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 02:00:11.076517 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076505 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-metric" Jul 21 02:00:11.076556 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076519 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-metric" Jul 21 02:00:11.076556 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076530 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-web" Jul 21 02:00:11.076556 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076535 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-web" Jul 21 02:00:11.076556 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076545 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="init-config-reloader" Jul 21 02:00:11.076556 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076550 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="init-config-reloader" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076558 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="config-reloader" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076565 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="config-reloader" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076572 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076577 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076584 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="alertmanager" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076589 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="alertmanager" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076600 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="prom-label-proxy" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076606 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="prom-label-proxy" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076650 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="alertmanager" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076658 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="config-reloader" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076665 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-web" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076672 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="prom-label-proxy" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076677 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy" Jul 21 02:00:11.076699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.076682 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" containerName="kube-rbac-proxy-metric" Jul 21 02:00:11.079913 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.079895 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.082271 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082254 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 02:00:11.082363 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082254 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 02:00:11.082363 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082255 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-92b9v\"" Jul 21 02:00:11.082363 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082255 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 02:00:11.082623 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082607 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 02:00:11.082696 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 02:00:11.082696 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082638 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 02:00:11.082919 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.082905 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 02:00:11.083098 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.083080 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 02:00:11.088313 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.088294 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 02:00:11.095548 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.095528 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 02:00:11.245201 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245170 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245201 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d975bb-1a3f-45d8-8b88-2c3df49a8542-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-config-volume\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245295 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245311 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d975bb-1a3f-45d8-8b88-2c3df49a8542-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245327 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-web-config\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdh76\" (UniqueName: \"kubernetes.io/projected/82d975bb-1a3f-45d8-8b88-2c3df49a8542-kube-api-access-wdh76\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245375 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d975bb-1a3f-45d8-8b88-2c3df49a8542-config-out\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245390 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245684 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245452 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d975bb-1a3f-45d8-8b88-2c3df49a8542-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245684 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245509 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d975bb-1a3f-45d8-8b88-2c3df49a8542-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.245684 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.245532 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346737 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d975bb-1a3f-45d8-8b88-2c3df49a8542-config-out\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346967 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346967 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d975bb-1a3f-45d8-8b88-2c3df49a8542-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346967 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346818 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d975bb-1a3f-45d8-8b88-2c3df49a8542-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346967 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346842 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346967 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.346967 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.346886 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347301 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d975bb-1a3f-45d8-8b88-2c3df49a8542-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347357 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d975bb-1a3f-45d8-8b88-2c3df49a8542-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-config-volume\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d975bb-1a3f-45d8-8b88-2c3df49a8542-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347564 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-web-config\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.347625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdh76\" (UniqueName: \"kubernetes.io/projected/82d975bb-1a3f-45d8-8b88-2c3df49a8542-kube-api-access-wdh76\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.348390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.348373 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d975bb-1a3f-45d8-8b88-2c3df49a8542-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.349071 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.349042 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d975bb-1a3f-45d8-8b88-2c3df49a8542-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.350360 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.350330 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d975bb-1a3f-45d8-8b88-2c3df49a8542-config-out\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351011 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.350686 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351011 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.350717 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d975bb-1a3f-45d8-8b88-2c3df49a8542-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351011 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.350874 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351011 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.350920 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351011 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.350928 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-config-volume\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351011 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.351007 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.351545 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.351530 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.352459 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.352443 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d975bb-1a3f-45d8-8b88-2c3df49a8542-web-config\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.354983 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.354963 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdh76\" (UniqueName: \"kubernetes.io/projected/82d975bb-1a3f-45d8-8b88-2c3df49a8542-kube-api-access-wdh76\") pod \"alertmanager-main-0\" (UID: \"82d975bb-1a3f-45d8-8b88-2c3df49a8542\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.389987 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.389963 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 02:00:11.521569 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:11.521548 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 02:00:11.523749 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:00:11.523714 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d975bb_1a3f_45d8_8b88_2c3df49a8542.slice/crio-9137c71cd0fcbd1ac13b3e2ffb571cfb14dbe9507f77f4b95c24db6e99ec67a0 WatchSource:0}: Error finding container 9137c71cd0fcbd1ac13b3e2ffb571cfb14dbe9507f77f4b95c24db6e99ec67a0: Status 404 returned error can't find the container with id 9137c71cd0fcbd1ac13b3e2ffb571cfb14dbe9507f77f4b95c24db6e99ec67a0 Jul 21 02:00:12.029956 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:12.029864 2561 generic.go:356] "Generic (PLEG): container finished" podID="82d975bb-1a3f-45d8-8b88-2c3df49a8542" containerID="b6e2ce585133be6de529b526b6d3f8a1b834302049763e19d08a65a3e5bf67ac" exitCode=0 Jul 21 02:00:12.030484 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:12.029951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerDied","Data":"b6e2ce585133be6de529b526b6d3f8a1b834302049763e19d08a65a3e5bf67ac"} Jul 21 02:00:12.030484 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:12.030016 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"9137c71cd0fcbd1ac13b3e2ffb571cfb14dbe9507f77f4b95c24db6e99ec67a0"} Jul 21 02:00:12.708390 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:12.708359 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51ae7178-9dbb-4892-b80c-49994c109d87" path="/var/lib/kubelet/pods/51ae7178-9dbb-4892-b80c-49994c109d87/volumes" Jul 21 02:00:13.038413 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.038314 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"942fbd85c05f59d395e0e782285d38c13ab647eec786933b8385b56ddac2501c"} Jul 21 02:00:13.038413 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.038380 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"81786f8423753973ab185314bc40f92f0201073661385d52eee8ed4cf6efa3cf"} Jul 21 02:00:13.038797 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.038426 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"2ad849ffd67718579dd47d601258bddb535c4e0685c48f2a1b6dc8004b7e093a"} Jul 21 02:00:13.038797 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.038456 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"d92f7a2fc2f7bce99352524dd8812f464bfbbdafaa4a1712e92d380b31d0ddd6"} Jul 21 02:00:13.038797 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.038482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"909b90c31bc651814303481b8e92ff56b19a9b6eeecf11daf6fac4eb67268160"} Jul 21 02:00:13.038797 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.038505 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d975bb-1a3f-45d8-8b88-2c3df49a8542","Type":"ContainerStarted","Data":"127b26bd1d7ef264043166afdee988f799b5686aa0fbb02d4b898fff3259ce79"} Jul 21 02:00:13.062874 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.062820 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.047596275 podStartE2EDuration="2.062805144s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.208869ms" isStatefulPod=false podCreationTimestamp="2026-07-21 02:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:00:13.06159998 +0000 UTC m=+400.878634617" watchObservedRunningTime="2026-07-21 02:00:13.062805144 +0000 UTC m=+400.879839781" Jul 21 02:00:13.185111 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.185075 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:13.291289 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.291165 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:13.356515 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.356482 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-bc79455f5-4f2ws"] Jul 21 02:00:13.359658 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.359587 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.361816 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.361786 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 21 02:00:13.361816 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.361793 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 21 02:00:13.362008 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.361820 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 21 02:00:13.362391 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.362373 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 21 02:00:13.362547 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.362527 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 21 02:00:13.362660 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.362647 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6g977\"" Jul 21 02:00:13.368102 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.368064 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 21 02:00:13.373182 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.373162 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bc79455f5-4f2ws"] Jul 21 02:00:13.468379 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8wl7\" (UniqueName: \"kubernetes.io/projected/55136372-a76c-4e35-8ccc-0159f80db161-kube-api-access-g8wl7\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468576 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-serving-certs-ca-bundle\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468576 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468576 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-secret-telemeter-client\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468576 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468532 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-telemeter-client-tls\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468576 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468555 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-federate-client-tls\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468645 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-metrics-client-ca\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.468850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.468745 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569197 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-serving-certs-ca-bundle\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569197 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569197 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569176 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-secret-telemeter-client\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569516 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569211 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-telemeter-client-tls\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569516 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-federate-client-tls\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569516 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569310 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-metrics-client-ca\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569516 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.569722 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8wl7\" (UniqueName: \"kubernetes.io/projected/55136372-a76c-4e35-8ccc-0159f80db161-kube-api-access-g8wl7\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.570014 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.569984 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-serving-certs-ca-bundle\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.570133 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.570021 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-metrics-client-ca\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.570233 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.570211 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55136372-a76c-4e35-8ccc-0159f80db161-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.571886 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.571866 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-secret-telemeter-client\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.571995 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.571943 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.572321 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.572303 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-federate-client-tls\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.572388 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.572337 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/55136372-a76c-4e35-8ccc-0159f80db161-telemeter-client-tls\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.577566 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.577547 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8wl7\" (UniqueName: \"kubernetes.io/projected/55136372-a76c-4e35-8ccc-0159f80db161-kube-api-access-g8wl7\") pod \"telemeter-client-bc79455f5-4f2ws\" (UID: \"55136372-a76c-4e35-8ccc-0159f80db161\") " pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.650433 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.649623 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 02:00:13.650433 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.649959 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="prometheus" containerID="cri-o://892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37" gracePeriod=600 Jul 21 02:00:13.650433 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.650310 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546" gracePeriod=600 Jul 21 02:00:13.650683 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.650437 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy" containerID="cri-o://a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4" gracePeriod=600 Jul 21 02:00:13.650683 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.650477 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="thanos-sidecar" containerID="cri-o://4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a" gracePeriod=600 Jul 21 02:00:13.650683 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.650504 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-web" containerID="cri-o://504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455" gracePeriod=600 Jul 21 02:00:13.650683 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.650513 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="config-reloader" containerID="cri-o://f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e" gracePeriod=600 Jul 21 02:00:13.672062 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.672027 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" Jul 21 02:00:13.828883 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:13.828851 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bc79455f5-4f2ws"] Jul 21 02:00:13.830506 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:00:13.830477 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55136372_a76c_4e35_8ccc_0159f80db161.slice/crio-cc56ac4511fb02f0961ab64e158de27566956192897dee29dc6bbcfa243e6a7f WatchSource:0}: Error finding container cc56ac4511fb02f0961ab64e158de27566956192897dee29dc6bbcfa243e6a7f: Status 404 returned error can't find the container with id cc56ac4511fb02f0961ab64e158de27566956192897dee29dc6bbcfa243e6a7f Jul 21 02:00:14.045898 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.045863 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546" exitCode=0 Jul 21 02:00:14.045898 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.045888 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4" exitCode=0 Jul 21 02:00:14.045898 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.045895 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a" exitCode=0 Jul 21 02:00:14.045898 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.045901 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e" exitCode=0 Jul 21 02:00:14.045898 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.045907 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37" exitCode=0 Jul 21 02:00:14.046463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.045930 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546"} Jul 21 02:00:14.046463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.046001 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4"} Jul 21 02:00:14.046463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.046028 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a"} Jul 21 02:00:14.046463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.046053 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e"} Jul 21 02:00:14.046463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.046077 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37"} Jul 21 02:00:14.047104 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.047083 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" event={"ID":"55136372-a76c-4e35-8ccc-0159f80db161","Type":"ContainerStarted","Data":"cc56ac4511fb02f0961ab64e158de27566956192897dee29dc6bbcfa243e6a7f"} Jul 21 02:00:14.887623 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.887595 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:14.984880 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.984847 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-tls\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985045 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.984892 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-metrics-client-certs\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985045 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.984920 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-web-config\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985045 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.984965 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-kubelet-serving-ca-bundle\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985045 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985000 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-grpc-tls\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985045 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985043 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985070 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-config-out\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985094 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-trusted-ca-bundle\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985131 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-kube-rbac-proxy\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985160 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-1\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985186 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-serving-certs-ca-bundle\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985211 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-thanos-prometheus-http-client-file\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985257 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985241 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-2\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985287 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhzv2\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-kube-api-access-nhzv2\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985319 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985360 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-config\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985436 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-metrics-client-ca\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985474 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-db\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985512 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-tls-assets\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985535 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-0\") pod \"25adda6b-afb9-4e1f-81a0-ff7504f59004\" (UID: \"25adda6b-afb9-4e1f-81a0-ff7504f59004\") " Jul 21 02:00:14.985621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985578 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.986000 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.985856 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:14.986500 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.986473 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.986500 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.986475 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.986947 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.986921 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.987377 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.987292 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:00:14.987490 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.987439 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.988032 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.988004 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.988162 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.988140 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 02:00:14.988809 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.988782 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.990111 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.990073 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.990221 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.990199 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-kube-api-access-nhzv2" (OuterVolumeSpecName: "kube-api-access-nhzv2") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "kube-api-access-nhzv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 02:00:14.990379 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.990344 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.991213 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.991178 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.991648 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.991616 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.991844 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.991805 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-config-out" (OuterVolumeSpecName: "config-out") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:00:14.991968 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.991913 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.992035 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.991983 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-config" (OuterVolumeSpecName: "config") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:14.992432 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.992384 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 02:00:14.992432 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:14.992420 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:15.002685 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.002649 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-web-config" (OuterVolumeSpecName: "web-config") pod "25adda6b-afb9-4e1f-81a0-ff7504f59004" (UID: "25adda6b-afb9-4e1f-81a0-ff7504f59004"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:00:15.054531 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.054505 2561 generic.go:356] "Generic (PLEG): container finished" podID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerID="504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455" exitCode=0 Jul 21 02:00:15.054850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.054550 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455"} Jul 21 02:00:15.054850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.054584 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.054850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.054604 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"25adda6b-afb9-4e1f-81a0-ff7504f59004","Type":"ContainerDied","Data":"1023c4beae046135aeb49ca602487cc6be029f92c8d2e2ba9636d3a15620c597"} Jul 21 02:00:15.054850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.054636 2561 scope.go:117] "RemoveContainer" containerID="9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546" Jul 21 02:00:15.062970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.062954 2561 scope.go:117] "RemoveContainer" containerID="a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4" Jul 21 02:00:15.069707 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.069692 2561 scope.go:117] "RemoveContainer" containerID="504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455" Jul 21 02:00:15.076441 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.076400 2561 scope.go:117] "RemoveContainer" containerID="4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a" Jul 21 02:00:15.077541 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.077512 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 02:00:15.082209 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.082191 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 02:00:15.083432 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.083415 2561 scope.go:117] "RemoveContainer" containerID="f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e" Jul 21 02:00:15.086690 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086670 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086756 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086692 2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-grpc-tls\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086756 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086702 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086756 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086711 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-config-out\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086756 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086722 2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-kube-rbac-proxy\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086756 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086735 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086756 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086748 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086762 2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086777 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086790 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhzv2\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-kube-api-access-nhzv2\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086806 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086821 2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-config\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086830 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-configmap-metrics-client-ca\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086839 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-db\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086848 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/25adda6b-afb9-4e1f-81a0-ff7504f59004-tls-assets\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086855 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/25adda6b-afb9-4e1f-81a0-ff7504f59004-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086863 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086871 2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-secret-metrics-client-certs\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.086952 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.086879 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/25adda6b-afb9-4e1f-81a0-ff7504f59004-web-config\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:00:15.089883 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.089869 2561 scope.go:117] "RemoveContainer" containerID="892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37" Jul 21 02:00:15.097584 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.097554 2561 scope.go:117] "RemoveContainer" containerID="1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91" Jul 21 02:00:15.105030 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105013 2561 scope.go:117] "RemoveContainer" containerID="9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546" Jul 21 02:00:15.105329 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.105304 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546\": container with ID starting with 9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546 not found: ID does not exist" containerID="9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546" Jul 21 02:00:15.105428 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105341 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546"} err="failed to get container status \"9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546\": rpc error: code = NotFound desc = could not find container \"9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546\": container with ID starting with 9f57867e5bb7baa1f474dd09a66c1bea27ee5041821296a0b817833b887f9546 not found: ID does not exist" Jul 21 02:00:15.105428 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105364 2561 scope.go:117] "RemoveContainer" containerID="a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4" Jul 21 02:00:15.105687 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.105665 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4\": container with ID starting with a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4 not found: ID does not exist" containerID="a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4" Jul 21 02:00:15.105768 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105693 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4"} err="failed to get container status \"a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4\": rpc error: code = NotFound desc = could not find container \"a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4\": container with ID starting with a13ef364f29da75cd4092210152ccd148a08e9836f3d9c09a8b324b0169c14b4 not found: ID does not exist" Jul 21 02:00:15.105768 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105716 2561 scope.go:117] "RemoveContainer" containerID="504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455" Jul 21 02:00:15.105939 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.105914 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455\": container with ID starting with 504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455 not found: ID does not exist" containerID="504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455" Jul 21 02:00:15.106052 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105942 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455"} err="failed to get container status \"504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455\": rpc error: code = NotFound desc = could not find container \"504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455\": container with ID starting with 504c5732829a9d2313fd718913cefb88bde9ea4f3d1b9e3a34c351dfe4885455 not found: ID does not exist" Jul 21 02:00:15.106052 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.105962 2561 scope.go:117] "RemoveContainer" containerID="4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a" Jul 21 02:00:15.106201 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.106179 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a\": container with ID starting with 4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a not found: ID does not exist" containerID="4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a" Jul 21 02:00:15.106282 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.106206 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a"} err="failed to get container status \"4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a\": rpc error: code = NotFound desc = could not find container \"4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a\": container with ID starting with 4e1549d95ee30534bc35601d095c5dce004dfbf7e811107106f78cb9063ea69a not found: ID does not exist" Jul 21 02:00:15.106282 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.106221 2561 scope.go:117] "RemoveContainer" containerID="f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e" Jul 21 02:00:15.106497 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.106473 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e\": container with ID starting with f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e not found: ID does not exist" containerID="f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e" Jul 21 02:00:15.106622 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.106505 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e"} err="failed to get container status \"f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e\": rpc error: code = NotFound desc = could not find container \"f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e\": container with ID starting with f72d07d07a2dfe04b02e704a78b8aaade1ce506766d13c3b0962c7945794596e not found: ID does not exist" Jul 21 02:00:15.106622 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.106527 2561 scope.go:117] "RemoveContainer" containerID="892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37" Jul 21 02:00:15.107224 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.106956 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37\": container with ID starting with 892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37 not found: ID does not exist" containerID="892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37" Jul 21 02:00:15.107224 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.107122 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37"} err="failed to get container status \"892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37\": rpc error: code = NotFound desc = could not find container \"892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37\": container with ID starting with 892f90e7058685235c282ee8a5a2505c7fd5509461ae5252682c038df925ab37 not found: ID does not exist" Jul 21 02:00:15.107224 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.107141 2561 scope.go:117] "RemoveContainer" containerID="1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91" Jul 21 02:00:15.107809 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:15.107790 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91\": container with ID starting with 1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91 not found: ID does not exist" containerID="1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91" Jul 21 02:00:15.107899 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.107814 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91"} err="failed to get container status \"1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91\": rpc error: code = NotFound desc = could not find container \"1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91\": container with ID starting with 1d5e157c484e30d35a6b4c4abd36bca7ed625da3577b2b3ca76e6f432761ab91 not found: ID does not exist" Jul 21 02:00:15.109275 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109256 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 02:00:15.109727 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109711 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="thanos-sidecar" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109730 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="thanos-sidecar" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109742 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-web" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109751 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-web" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109765 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="prometheus" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109774 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="prometheus" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109787 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="init-config-reloader" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109795 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="init-config-reloader" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109805 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-thanos" Jul 21 02:00:15.109812 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109814 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-thanos" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109836 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109844 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109860 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="config-reloader" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109868 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="config-reloader" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109934 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-thanos" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109945 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="prometheus" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109953 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109961 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="kube-rbac-proxy-web" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109972 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="thanos-sidecar" Jul 21 02:00:15.110142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.109981 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" containerName="config-reloader" Jul 21 02:00:15.114197 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.114175 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.117039 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.117019 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 21 02:00:15.117122 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.117111 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 21 02:00:15.117725 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.117491 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 21 02:00:15.117725 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.117702 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ctz2d\"" Jul 21 02:00:15.118469 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118144 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 21 02:00:15.118469 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118224 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 21 02:00:15.118469 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118296 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 21 02:00:15.118469 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118396 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 21 02:00:15.118723 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118421 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 21 02:00:15.118916 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118895 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 21 02:00:15.118989 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.118969 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 21 02:00:15.119121 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.119098 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6m4bdd03ghitu\"" Jul 21 02:00:15.119312 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.119261 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 21 02:00:15.119312 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.119263 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 21 02:00:15.121364 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.121343 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 21 02:00:15.125093 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.125074 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 21 02:00:15.133661 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.133640 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 02:00:15.288123 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288083 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288123 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288127 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288342 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288342 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288342 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288230 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288342 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288260 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cff4d7dc-c42d-4129-a7de-5e456da7465d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288342 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288321 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288438 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288491 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cff4d7dc-c42d-4129-a7de-5e456da7465d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288611 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288602 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288650 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288666 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288718 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288748 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-config\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288770 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.288818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.288795 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk5ss\" (UniqueName: \"kubernetes.io/projected/cff4d7dc-c42d-4129-a7de-5e456da7465d-kube-api-access-qk5ss\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389529 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cff4d7dc-c42d-4129-a7de-5e456da7465d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389529 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389495 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389529 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389516 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389792 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389792 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389792 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389677 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cff4d7dc-c42d-4129-a7de-5e456da7465d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389792 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389703 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389792 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389738 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.389792 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389837 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389881 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389916 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389923 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-config\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.389978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.390031 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk5ss\" (UniqueName: \"kubernetes.io/projected/cff4d7dc-c42d-4129-a7de-5e456da7465d-kube-api-access-qk5ss\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.390069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390117 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.390094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390602 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.390123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390602 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.390154 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.390602 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.390185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.391200 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.391171 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.391313 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.391212 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393502 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.392788 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393502 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.392936 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393502 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393042 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393502 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393382 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393760 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393542 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-config\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393760 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393651 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cff4d7dc-c42d-4129-a7de-5e456da7465d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393760 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393668 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393909 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393838 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.393909 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.393844 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-web-config\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.394384 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.394339 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.395212 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.395149 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.395903 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.395881 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cff4d7dc-c42d-4129-a7de-5e456da7465d-config-out\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.396206 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.396174 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.396557 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.396536 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.401477 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.397456 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cff4d7dc-c42d-4129-a7de-5e456da7465d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.401477 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.397588 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cff4d7dc-c42d-4129-a7de-5e456da7465d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.403673 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.403648 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk5ss\" (UniqueName: \"kubernetes.io/projected/cff4d7dc-c42d-4129-a7de-5e456da7465d-kube-api-access-qk5ss\") pod \"prometheus-k8s-0\" (UID: \"cff4d7dc-c42d-4129-a7de-5e456da7465d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.427264 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.427234 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:15.570041 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:15.570000 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 02:00:15.573693 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:00:15.573666 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff4d7dc_c42d_4129_a7de_5e456da7465d.slice/crio-97046ae1d9d39ba1767076567deb00aaed33b5b82a0b4d9b49b84c3707d9f4c2 WatchSource:0}: Error finding container 97046ae1d9d39ba1767076567deb00aaed33b5b82a0b4d9b49b84c3707d9f4c2: Status 404 returned error can't find the container with id 97046ae1d9d39ba1767076567deb00aaed33b5b82a0b4d9b49b84c3707d9f4c2 Jul 21 02:00:16.062192 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:16.062113 2561 generic.go:356] "Generic (PLEG): container finished" podID="cff4d7dc-c42d-4129-a7de-5e456da7465d" containerID="7a64fa29c3f960609e98e33e159f9485d864d20ebad37aae8042d0d0c4bad27c" exitCode=0 Jul 21 02:00:16.062729 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:16.062201 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerDied","Data":"7a64fa29c3f960609e98e33e159f9485d864d20ebad37aae8042d0d0c4bad27c"} Jul 21 02:00:16.062729 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:16.062309 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"97046ae1d9d39ba1767076567deb00aaed33b5b82a0b4d9b49b84c3707d9f4c2"} Jul 21 02:00:16.708788 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:16.708757 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25adda6b-afb9-4e1f-81a0-ff7504f59004" path="/var/lib/kubelet/pods/25adda6b-afb9-4e1f-81a0-ff7504f59004/volumes" Jul 21 02:00:17.069328 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.069290 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" event={"ID":"55136372-a76c-4e35-8ccc-0159f80db161","Type":"ContainerStarted","Data":"df6141671e7c5782580e80f63aab28b4744f3fc9c4fa30362d2c5df5e104c088"} Jul 21 02:00:17.069793 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.069360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" event={"ID":"55136372-a76c-4e35-8ccc-0159f80db161","Type":"ContainerStarted","Data":"be5640bdcf55eeef2e6f402061809e41578d6eac5e11b9aa53d77888b7f13d4d"} Jul 21 02:00:17.069793 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.069388 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" event={"ID":"55136372-a76c-4e35-8ccc-0159f80db161","Type":"ContainerStarted","Data":"67573d614c0d416d7dc412610ac3bca54aca63c2708519d531ac3320108664f5"} Jul 21 02:00:17.072306 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.072283 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"8022c7a627129a893a5e4a8704de8d1b6807aa757c0e09df3083a4810328c418"} Jul 21 02:00:17.072448 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.072346 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"3f6dd768596b3c9c641c23981d5dc6bc03ae07f6195197c30a9f1c6413027484"} Jul 21 02:00:17.072448 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.072385 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"dc8f135b2ea573d986eb19ccb98024e7235f2bbffb332684ebdce365c36e96ed"} Jul 21 02:00:17.072448 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.072446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"0b8e2a76b0545fc872f52760e53cc35d28eb3b0b5b435201f9b83d39ea21b616"} Jul 21 02:00:17.072600 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.072483 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"a8f4a95fc335e7b048851c2fef8e10fc0d99c0a713b01c2445a52000b2454f0e"} Jul 21 02:00:17.072600 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.072529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cff4d7dc-c42d-4129-a7de-5e456da7465d","Type":"ContainerStarted","Data":"54edae068667a5d710074666fbb70132e76b48fea652d16e5817afbbbb3436f6"} Jul 21 02:00:17.101294 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.101248 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-bc79455f5-4f2ws" podStartSLOduration=1.615250812 podStartE2EDuration="4.101234509s" totalImagesPullingTime="2.485983697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:00:17.100133939 +0000 UTC m=+404.917168578" watchObservedRunningTime="2026-07-21 02:00:17.101234509 +0000 UTC m=+404.918269195" Jul 21 02:00:17.130415 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:17.130357 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.10831182 podStartE2EDuration="2.130343891s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.032071ms" isStatefulPod=false podCreationTimestamp="2026-07-21 02:00:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:00:17.128478797 +0000 UTC m=+404.945513435" watchObservedRunningTime="2026-07-21 02:00:17.130343891 +0000 UTC m=+404.947378529" Jul 21 02:00:18.493537 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:18.493486 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" podUID="d5e054ae-c8cc-41cf-b425-41ed5e1206ff" Jul 21 02:00:18.493900 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:00:18.493642 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7hq8m" podUID="42ad014a-085f-4801-9869-6f03c196609a" Jul 21 02:00:19.004334 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:19.004271 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:19.057514 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:19.057479 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:19.080643 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:19.080619 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hq8m" Jul 21 02:00:19.080751 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:19.080620 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 02:00:20.427985 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.427941 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:00:20.434031 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.434001 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 02:00:20.434138 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.434056 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 02:00:20.436433 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.436386 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d5e054ae-c8cc-41cf-b425-41ed5e1206ff-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-svqv9\" (UID: \"d5e054ae-c8cc-41cf-b425-41ed5e1206ff\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 02:00:20.436538 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.436462 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/42ad014a-085f-4801-9869-6f03c196609a-metrics-tls\") pod \"dns-default-7hq8m\" (UID: \"42ad014a-085f-4801-9869-6f03c196609a\") " pod="openshift-dns/dns-default-7hq8m" Jul 21 02:00:20.583862 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.583835 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l4zpp\"" Jul 21 02:00:20.584200 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.584185 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cbx74\"" Jul 21 02:00:20.592640 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.592620 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hq8m" Jul 21 02:00:20.592640 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.592633 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" Jul 21 02:00:20.731370 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.731342 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hq8m"] Jul 21 02:00:20.733286 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:00:20.733261 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ad014a_085f_4801_9869_6f03c196609a.slice/crio-0aa1dfa98c56d8b534e3493fc4ad54d3a336fe58fdb039ad6df3981c9917fb27 WatchSource:0}: Error finding container 0aa1dfa98c56d8b534e3493fc4ad54d3a336fe58fdb039ad6df3981c9917fb27: Status 404 returned error can't find the container with id 0aa1dfa98c56d8b534e3493fc4ad54d3a336fe58fdb039ad6df3981c9917fb27 Jul 21 02:00:20.746713 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:20.746690 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9"] Jul 21 02:00:20.749046 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:00:20.749021 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e054ae_c8cc_41cf_b425_41ed5e1206ff.slice/crio-682f9a21568919acfa0e33b4c78b82448ec47f1c4dc47e3eaa5ac17ee9713b95 WatchSource:0}: Error finding container 682f9a21568919acfa0e33b4c78b82448ec47f1c4dc47e3eaa5ac17ee9713b95: Status 404 returned error can't find the container with id 682f9a21568919acfa0e33b4c78b82448ec47f1c4dc47e3eaa5ac17ee9713b95 Jul 21 02:00:21.090654 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:21.090624 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" event={"ID":"d5e054ae-c8cc-41cf-b425-41ed5e1206ff","Type":"ContainerStarted","Data":"682f9a21568919acfa0e33b4c78b82448ec47f1c4dc47e3eaa5ac17ee9713b95"} Jul 21 02:00:21.091707 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:21.091683 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hq8m" event={"ID":"42ad014a-085f-4801-9869-6f03c196609a","Type":"ContainerStarted","Data":"0aa1dfa98c56d8b534e3493fc4ad54d3a336fe58fdb039ad6df3981c9917fb27"} Jul 21 02:00:22.099700 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:22.099667 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" event={"ID":"d5e054ae-c8cc-41cf-b425-41ed5e1206ff","Type":"ContainerStarted","Data":"ea25b577ba699d9109cf3bb1bdbe87b20f38a5ddd5a38cbb12d64b5e23b853fb"} Jul 21 02:00:22.115942 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:22.115894 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-svqv9" podStartSLOduration=382.869821268 podStartE2EDuration="6m24.115873388s" totalImagesPullingTime="1.24605212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:00:22.11412063 +0000 UTC m=+409.931155270" watchObservedRunningTime="2026-07-21 02:00:22.115873388 +0000 UTC m=+409.932908027" Jul 21 02:00:22.218864 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:22.218824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:22.286528 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:22.286451 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:23.105724 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:23.105692 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hq8m" event={"ID":"42ad014a-085f-4801-9869-6f03c196609a","Type":"ContainerStarted","Data":"9af703373d4e331deb2dc50e7a06e03297dbd5cd9df99175e17c15501494b861"} Jul 21 02:00:23.106086 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:23.105755 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hq8m" event={"ID":"42ad014a-085f-4801-9869-6f03c196609a","Type":"ContainerStarted","Data":"1a896f5a057645e28d66f13e3adadf8a1d2a4f8e1be5aad81f6f7243f22a6c37"} Jul 21 02:00:23.123721 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:23.123678 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7hq8m" podStartSLOduration=373.233998721 podStartE2EDuration="6m15.123662559s" totalImagesPullingTime="1.889663838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 01:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:00:23.121625021 +0000 UTC m=+410.938659658" watchObservedRunningTime="2026-07-21 02:00:23.123662559 +0000 UTC m=+410.940697196" Jul 21 02:00:24.109880 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:24.109853 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7hq8m" Jul 21 02:00:28.557508 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:28.557475 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:28.658862 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:28.658829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:28.711935 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:28.711908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:34.116453 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:34.116395 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7hq8m" Jul 21 02:00:43.241047 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:43.241006 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:00:43.341596 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:00:43.341569 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:05.057581 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:05.057509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:10.936162 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:10.936117 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:10.991662 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:10.991630 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:15.427706 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:15.427621 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:01:15.443085 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:15.443040 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:01:16.345174 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:16.345143 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 02:01:23.528587 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:23.528549 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:23.590565 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:23.590536 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:31.904697 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:31.904655 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:31.977645 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:31.977614 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:35.411632 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:35.411596 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:35.463723 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:35.463692 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:42.898306 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:42.898266 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:43.001100 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:43.001072 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:43.078860 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:43.078829 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:50.104514 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:50.104480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:50.205453 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:50.205396 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:01:50.404173 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:01:50.404091 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:02:13.063203 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:02:13.063165 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:02:13.118042 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:02:13.118014 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:02:31.531526 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:02:31.531435 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:02:44.823115 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:02:44.823074 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:02:45.783955 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:02:45.783897 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:13.133022 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:13.132985 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:14.606369 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:14.606332 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:16.985628 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:16.985591 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:17.103818 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:17.103783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:18.585059 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:18.585022 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:18.817850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:18.817806 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:19.307392 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:19.307359 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:20.877752 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:20.877719 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:21.211188 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:21.211102 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:24.404870 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:24.404836 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:24.535055 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:24.535023 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:27.320449 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:27.320394 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:27.458813 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:27.458783 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:29.316801 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:29.316765 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:31.063878 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:31.063843 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:32.401565 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:32.401529 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:33.905217 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:33.905190 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:03:33.906863 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:33.906840 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:03:34.277272 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:34.277187 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:36.946919 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:36.946882 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:38.464085 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:38.464044 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:39.165001 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:39.164953 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:53.617869 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:53.617824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:53.723861 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:53.723824 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:54.225993 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.225959 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:54.672664 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.672625 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:54.778987 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.778956 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:54.843510 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.843486 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684"] Jul 21 02:03:54.845717 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.845702 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.847998 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.847976 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 02:03:54.848447 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.848427 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rnskh\"" Jul 21 02:03:54.848447 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.848442 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 02:03:54.848606 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.848484 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 02:03:54.848606 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.848523 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 02:03:54.865862 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.865840 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684"] Jul 21 02:03:54.895337 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.895305 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8594d600-acc0-4220-b7d7-c407fe69641d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.895466 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.895353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgfxp\" (UniqueName: \"kubernetes.io/projected/8594d600-acc0-4220-b7d7-c407fe69641d-kube-api-access-mgfxp\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.895466 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.895386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8594d600-acc0-4220-b7d7-c407fe69641d-tmp\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.895466 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.895435 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8594d600-acc0-4220-b7d7-c407fe69641d-manifests\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.895466 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.895456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8594d600-acc0-4220-b7d7-c407fe69641d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.996419 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.996333 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8594d600-acc0-4220-b7d7-c407fe69641d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.996419 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.996376 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgfxp\" (UniqueName: \"kubernetes.io/projected/8594d600-acc0-4220-b7d7-c407fe69641d-kube-api-access-mgfxp\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.996419 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.996398 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8594d600-acc0-4220-b7d7-c407fe69641d-tmp\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.996646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.996456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8594d600-acc0-4220-b7d7-c407fe69641d-manifests\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.996646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.996489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8594d600-acc0-4220-b7d7-c407fe69641d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.996874 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.996847 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8594d600-acc0-4220-b7d7-c407fe69641d-manifests\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.998531 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.998513 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8594d600-acc0-4220-b7d7-c407fe69641d-tmp\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.998802 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.998782 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8594d600-acc0-4220-b7d7-c407fe69641d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:54.998863 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:54.998815 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8594d600-acc0-4220-b7d7-c407fe69641d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:55.005427 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.005392 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.005645 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.005629 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgfxp\" (UniqueName: \"kubernetes.io/projected/8594d600-acc0-4220-b7d7-c407fe69641d-kube-api-access-mgfxp\") pod \"opendatahub-operator-controller-manager-5c97f6b6f9-56684\" (UID: \"8594d600-acc0-4220-b7d7-c407fe69641d\") " pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:55.105830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.105808 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.156069 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.156047 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:03:55.204346 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.204299 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.270624 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.270523 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.288514 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.288490 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684"] Jul 21 02:03:55.291953 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:03:55.291926 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8594d600_acc0_4220_b7d7_c407fe69641d.slice/crio-79a7aa573631e0509a3246303ef9170464c814386ef74c022e0af638cd3402bc WatchSource:0}: Error finding container 79a7aa573631e0509a3246303ef9170464c814386ef74c022e0af638cd3402bc: Status 404 returned error can't find the container with id 79a7aa573631e0509a3246303ef9170464c814386ef74c022e0af638cd3402bc Jul 21 02:03:55.293701 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.293684 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 02:03:55.331714 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.331687 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.432895 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.432869 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.536556 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.536490 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:03:55.976030 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:55.976000 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" event={"ID":"8594d600-acc0-4220-b7d7-c407fe69641d","Type":"ContainerStarted","Data":"79a7aa573631e0509a3246303ef9170464c814386ef74c022e0af638cd3402bc"} Jul 21 02:03:59.997539 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:03:59.997499 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" event={"ID":"8594d600-acc0-4220-b7d7-c407fe69641d","Type":"ContainerStarted","Data":"6adbd82d47bb9516d62aa38bdfbf12729716ca54e97f956beda245b367161169"} Jul 21 02:04:01.003428 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:01.003376 2561 generic.go:356] "Generic (PLEG): container finished" podID="8594d600-acc0-4220-b7d7-c407fe69641d" containerID="6adbd82d47bb9516d62aa38bdfbf12729716ca54e97f956beda245b367161169" exitCode=0 Jul 21 02:04:01.003827 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:01.003444 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" event={"ID":"8594d600-acc0-4220-b7d7-c407fe69641d","Type":"ContainerDied","Data":"6adbd82d47bb9516d62aa38bdfbf12729716ca54e97f956beda245b367161169"} Jul 21 02:04:02.010144 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:02.010111 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" event={"ID":"8594d600-acc0-4220-b7d7-c407fe69641d","Type":"ContainerStarted","Data":"a08cd32504fd5e1ebf10ba4e8f9f3d4444228d3d8013f69c8dd024527d2fe17f"} Jul 21 02:04:02.010507 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:02.010245 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:04:02.033609 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:02.033570 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" podStartSLOduration=2.929276498 podStartE2EDuration="8.033554692s" totalImagesPullingTime="3.992674474s" totalInitContainerRuntime="1.11160372s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:03:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:04:02.030400658 +0000 UTC m=+629.847435307" watchObservedRunningTime="2026-07-21 02:04:02.033554692 +0000 UTC m=+629.850589330" Jul 21 02:04:10.031745 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:10.031708 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:10.838989 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:10.838952 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:13.016268 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.016239 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c97f6b6f9-56684" Jul 21 02:04:13.292545 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.292467 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf"] Jul 21 02:04:13.301586 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.301560 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.303568 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.303542 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 02:04:13.303686 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.303650 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 02:04:13.303686 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.303666 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 02:04:13.303802 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.303658 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vvhx2\"" Jul 21 02:04:13.303802 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.303750 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 02:04:13.304003 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.303989 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 02:04:13.306957 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.306934 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf"] Jul 21 02:04:13.343671 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.343647 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/baf9a2dc-661e-449d-8f8c-b5bed1085c95-manager-config\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.343789 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.343695 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/baf9a2dc-661e-449d-8f8c-b5bed1085c95-metrics-cert\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.343789 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.343750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf9a2dc-661e-449d-8f8c-b5bed1085c95-cert\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.343906 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.343817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgxkp\" (UniqueName: \"kubernetes.io/projected/baf9a2dc-661e-449d-8f8c-b5bed1085c95-kube-api-access-lgxkp\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.444641 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.444619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/baf9a2dc-661e-449d-8f8c-b5bed1085c95-manager-config\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.444779 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.444661 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/baf9a2dc-661e-449d-8f8c-b5bed1085c95-metrics-cert\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.444779 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.444697 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf9a2dc-661e-449d-8f8c-b5bed1085c95-cert\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.444779 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.444725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgxkp\" (UniqueName: \"kubernetes.io/projected/baf9a2dc-661e-449d-8f8c-b5bed1085c95-kube-api-access-lgxkp\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.445351 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.445333 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/baf9a2dc-661e-449d-8f8c-b5bed1085c95-manager-config\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.447065 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.447041 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf9a2dc-661e-449d-8f8c-b5bed1085c95-cert\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.447158 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.447119 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/baf9a2dc-661e-449d-8f8c-b5bed1085c95-metrics-cert\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.452102 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.452083 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgxkp\" (UniqueName: \"kubernetes.io/projected/baf9a2dc-661e-449d-8f8c-b5bed1085c95-kube-api-access-lgxkp\") pod \"lws-controller-manager-5dfc8579c-tnnbf\" (UID: \"baf9a2dc-661e-449d-8f8c-b5bed1085c95\") " pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.611272 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.611233 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:13.743873 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:13.743850 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf"] Jul 21 02:04:13.746197 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:04:13.746172 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf9a2dc_661e_449d_8f8c_b5bed1085c95.slice/crio-5796abb168a4ac7e156a5c719deed27317e8c6e8ccea31c0d655be3ef0865c01 WatchSource:0}: Error finding container 5796abb168a4ac7e156a5c719deed27317e8c6e8ccea31c0d655be3ef0865c01: Status 404 returned error can't find the container with id 5796abb168a4ac7e156a5c719deed27317e8c6e8ccea31c0d655be3ef0865c01 Jul 21 02:04:14.073857 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:14.073777 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" event={"ID":"baf9a2dc-661e-449d-8f8c-b5bed1085c95","Type":"ContainerStarted","Data":"5796abb168a4ac7e156a5c719deed27317e8c6e8ccea31c0d655be3ef0865c01"} Jul 21 02:04:14.750067 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:14.749978 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:15.338170 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.338132 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg"] Jul 21 02:04:15.350080 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.350057 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.353726 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.353695 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 02:04:15.354108 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.354083 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 02:04:15.354216 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.354148 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 02:04:15.354216 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.354178 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hznsg\"" Jul 21 02:04:15.355338 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.355316 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 02:04:15.360157 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.360134 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg"] Jul 21 02:04:15.463817 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.463787 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66487a6d-99f6-4531-a329-8a818c1d3860-tls-certs\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.463980 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.463829 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmzdz\" (UniqueName: \"kubernetes.io/projected/66487a6d-99f6-4531-a329-8a818c1d3860-kube-api-access-wmzdz\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.463980 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.463878 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66487a6d-99f6-4531-a329-8a818c1d3860-tmp\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.564830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.564791 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66487a6d-99f6-4531-a329-8a818c1d3860-tls-certs\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.565002 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.564846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmzdz\" (UniqueName: \"kubernetes.io/projected/66487a6d-99f6-4531-a329-8a818c1d3860-kube-api-access-wmzdz\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.565002 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.564896 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66487a6d-99f6-4531-a329-8a818c1d3860-tmp\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.567371 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.567344 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66487a6d-99f6-4531-a329-8a818c1d3860-tmp\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.567652 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.567633 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66487a6d-99f6-4531-a329-8a818c1d3860-tls-certs\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.583130 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.583105 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmzdz\" (UniqueName: \"kubernetes.io/projected/66487a6d-99f6-4531-a329-8a818c1d3860-kube-api-access-wmzdz\") pod \"kube-auth-proxy-5b99b8fb9b-j2lbg\" (UID: \"66487a6d-99f6-4531-a329-8a818c1d3860\") " pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.662995 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.662917 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" Jul 21 02:04:15.798677 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.798637 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:15.877825 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.877712 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg"] Jul 21 02:04:15.880175 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:04:15.880150 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66487a6d_99f6_4531_a329_8a818c1d3860.slice/crio-c97f2b0279f31c0b092c05b63d473fb0f0e02618cc772a6cdad365b79614057c WatchSource:0}: Error finding container c97f2b0279f31c0b092c05b63d473fb0f0e02618cc772a6cdad365b79614057c: Status 404 returned error can't find the container with id c97f2b0279f31c0b092c05b63d473fb0f0e02618cc772a6cdad365b79614057c Jul 21 02:04:15.943073 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:15.942977 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:16.085418 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.085383 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" event={"ID":"baf9a2dc-661e-449d-8f8c-b5bed1085c95","Type":"ContainerStarted","Data":"dd3b0870f9195d33338d340322642242e07f88d5618dab028bd55ee030c6ab9b"} Jul 21 02:04:16.085567 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.085479 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:16.086570 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.086548 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" event={"ID":"66487a6d-99f6-4531-a329-8a818c1d3860","Type":"ContainerStarted","Data":"c97f2b0279f31c0b092c05b63d473fb0f0e02618cc772a6cdad365b79614057c"} Jul 21 02:04:16.182236 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.182205 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:16.188479 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.188436 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" podStartSLOduration=1.130633739 podStartE2EDuration="3.188422384s" totalImagesPullingTime="2.057788645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:04:16.168813053 +0000 UTC m=+643.985847690" watchObservedRunningTime="2026-07-21 02:04:16.188422384 +0000 UTC m=+644.005457013" Jul 21 02:04:16.379702 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.379663 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:16.593586 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:16.593536 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:17.357981 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:17.357948 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:17.772178 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:17.772089 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:17.871220 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:17.871166 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:18.073430 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:18.073330 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:18.967935 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:18.967908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:19.101720 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:19.101690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" event={"ID":"66487a6d-99f6-4531-a329-8a818c1d3860","Type":"ContainerStarted","Data":"4fba275d41288a339da87d41d08eb5fa22d60747f06d389c4f6e9080985c1294"} Jul 21 02:04:19.120066 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:19.119994 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b99b8fb9b-j2lbg" podStartSLOduration=1.509434921 podStartE2EDuration="4.119979749s" totalImagesPullingTime="2.610544828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:04:19.118248551 +0000 UTC m=+646.935283189" watchObservedRunningTime="2026-07-21 02:04:19.119979749 +0000 UTC m=+646.937014390" Jul 21 02:04:23.569056 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:23.569013 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:23.743369 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:23.743338 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:24.520774 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.520739 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn"] Jul 21 02:04:24.524189 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.524170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.526455 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.526437 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-r5sqt\"" Jul 21 02:04:24.526700 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.526687 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 02:04:24.540466 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.540437 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn"] Jul 21 02:04:24.564559 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.564528 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:24.643621 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643597 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xtst\" (UniqueName: \"kubernetes.io/projected/23eb119c-4f4e-401c-8795-5336619b75c8-kube-api-access-5xtst\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23eb119c-4f4e-401c-8795-5336619b75c8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643687 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643740 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23eb119c-4f4e-401c-8795-5336619b75c8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643786 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643837 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.643970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.643903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23eb119c-4f4e-401c-8795-5336619b75c8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744599 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744571 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744733 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23eb119c-4f4e-401c-8795-5336619b75c8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744733 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744625 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744733 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744733 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744676 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744733 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744716 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23eb119c-4f4e-401c-8795-5336619b75c8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744982 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744756 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744982 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xtst\" (UniqueName: \"kubernetes.io/projected/23eb119c-4f4e-401c-8795-5336619b75c8-kube-api-access-5xtst\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.744982 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.744836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23eb119c-4f4e-401c-8795-5336619b75c8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.745135 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.745038 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.745135 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.745104 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.745242 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.745226 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.745399 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.745374 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.745557 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.745538 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23eb119c-4f4e-401c-8795-5336619b75c8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.747204 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.747185 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23eb119c-4f4e-401c-8795-5336619b75c8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.747279 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.747243 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23eb119c-4f4e-401c-8795-5336619b75c8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.762332 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.762305 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23eb119c-4f4e-401c-8795-5336619b75c8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.764458 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.764435 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xtst\" (UniqueName: \"kubernetes.io/projected/23eb119c-4f4e-401c-8795-5336619b75c8-kube-api-access-5xtst\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zzsbn\" (UID: \"23eb119c-4f4e-401c-8795-5336619b75c8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.834841 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.834773 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:24.914096 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.914065 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:24.982496 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:24.982428 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn"] Jul 21 02:04:24.984712 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:04:24.984681 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23eb119c_4f4e_401c_8795_5336619b75c8.slice/crio-b9f4ae9d2aa49fe83fd2a8b6a23df98565c6b4bbc949c1f346154d5fc088d219 WatchSource:0}: Error finding container b9f4ae9d2aa49fe83fd2a8b6a23df98565c6b4bbc949c1f346154d5fc088d219: Status 404 returned error can't find the container with id b9f4ae9d2aa49fe83fd2a8b6a23df98565c6b4bbc949c1f346154d5fc088d219 Jul 21 02:04:25.011779 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:25.011752 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:25.133778 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:25.133745 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" event={"ID":"23eb119c-4f4e-401c-8795-5336619b75c8","Type":"ContainerStarted","Data":"b9f4ae9d2aa49fe83fd2a8b6a23df98565c6b4bbc949c1f346154d5fc088d219"} Jul 21 02:04:26.562200 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:26.562159 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:27.093831 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:27.093789 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5dfc8579c-tnnbf" Jul 21 02:04:27.352245 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:27.352147 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:27.451942 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:27.451904 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:27.484840 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:27.484799 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:04:27.484942 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:27.484876 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:04:27.484942 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:27.484903 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:04:28.150577 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:28.150537 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" event={"ID":"23eb119c-4f4e-401c-8795-5336619b75c8","Type":"ContainerStarted","Data":"632aec2948043cf7abd4aaba15db1484c288618298210b1518ca5f20fd8660a4"} Jul 21 02:04:28.835571 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:28.835538 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:28.839796 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:28.839771 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:28.867012 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:28.866965 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" podStartSLOduration=2.369190348 podStartE2EDuration="4.866948699s" totalImagesPullingTime="2.497758351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:04:28.19922567 +0000 UTC m=+656.016260307" watchObservedRunningTime="2026-07-21 02:04:28.866948699 +0000 UTC m=+656.683983336" Jul 21 02:04:29.155744 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:29.155716 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:29.156618 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:29.156601 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zzsbn" Jul 21 02:04:44.674517 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:44.674486 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:04:44.725185 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:04:44.725152 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:03.780736 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:03.780703 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:03.881329 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:03.881298 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:05.728216 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:05.728182 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:05.786948 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:05.786908 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:16.387810 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:16.387780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:16.488488 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:16.488454 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:16.587205 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:16.587178 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:27.644510 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:27.644421 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:27.813142 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:27.813107 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:39.067103 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:39.067062 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:39.167191 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:39.167154 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:39.325861 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:39.325782 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:39.413959 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:39.413895 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:45.298435 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:45.298389 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:48.840198 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:48.840157 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:51.881917 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:51.881872 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:53.355697 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:53.355642 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:54.244549 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:54.244515 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:56.442872 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:56.442835 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:56.837583 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:56.837501 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:57.083827 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:57.083781 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:57.216990 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:57.216955 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:05:59.889397 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:05:59.889364 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:01.332750 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:01.332714 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:01.523686 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:01.523653 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:03.305700 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.305669 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c"] Jul 21 02:06:03.309099 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.309080 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.311007 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.310982 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 21 02:06:03.311124 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.311025 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 02:06:03.311537 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.311517 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tkr4w\"" Jul 21 02:06:03.311537 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.311526 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 02:06:03.311682 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.311516 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 21 02:06:03.316427 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.316391 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c"] Jul 21 02:06:03.358670 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.358640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ds62\" (UniqueName: \"kubernetes.io/projected/99231f30-37bc-4ae2-84e6-7ce19cccee07-kube-api-access-7ds62\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.358764 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.358681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/99231f30-37bc-4ae2-84e6-7ce19cccee07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.358764 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.358700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/99231f30-37bc-4ae2-84e6-7ce19cccee07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.365500 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.365478 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:03.459496 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.459469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ds62\" (UniqueName: \"kubernetes.io/projected/99231f30-37bc-4ae2-84e6-7ce19cccee07-kube-api-access-7ds62\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.459590 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.459501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/99231f30-37bc-4ae2-84e6-7ce19cccee07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.459590 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.459521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/99231f30-37bc-4ae2-84e6-7ce19cccee07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.459665 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:06:03.459610 2561 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 21 02:06:03.459717 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:06:03.459676 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99231f30-37bc-4ae2-84e6-7ce19cccee07-plugin-serving-cert podName:99231f30-37bc-4ae2-84e6-7ce19cccee07 nodeName:}" failed. No retries permitted until 2026-07-21 02:06:03.959661956 +0000 UTC m=+751.776696572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/99231f30-37bc-4ae2-84e6-7ce19cccee07-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-xzm5c" (UID: "99231f30-37bc-4ae2-84e6-7ce19cccee07") : secret "plugin-serving-cert" not found Jul 21 02:06:03.460185 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.460167 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/99231f30-37bc-4ae2-84e6-7ce19cccee07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.469828 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.469801 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ds62\" (UniqueName: \"kubernetes.io/projected/99231f30-37bc-4ae2-84e6-7ce19cccee07-kube-api-access-7ds62\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.555384 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.555355 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:03.963422 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.963381 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/99231f30-37bc-4ae2-84e6-7ce19cccee07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:03.965613 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:03.965582 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/99231f30-37bc-4ae2-84e6-7ce19cccee07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xzm5c\" (UID: \"99231f30-37bc-4ae2-84e6-7ce19cccee07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:04.228991 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:04.228921 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" Jul 21 02:06:04.352689 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:04.352662 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c"] Jul 21 02:06:04.354829 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:06:04.354799 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99231f30_37bc_4ae2_84e6_7ce19cccee07.slice/crio-364d85ce31c5482cc8ead0e4ba3d6e3d31b25eff73b7a0c4bfd9a47b865581d8 WatchSource:0}: Error finding container 364d85ce31c5482cc8ead0e4ba3d6e3d31b25eff73b7a0c4bfd9a47b865581d8: Status 404 returned error can't find the container with id 364d85ce31c5482cc8ead0e4ba3d6e3d31b25eff73b7a0c4bfd9a47b865581d8 Jul 21 02:06:04.593465 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:04.593434 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" event={"ID":"99231f30-37bc-4ae2-84e6-7ce19cccee07","Type":"ContainerStarted","Data":"364d85ce31c5482cc8ead0e4ba3d6e3d31b25eff73b7a0c4bfd9a47b865581d8"} Jul 21 02:06:04.841705 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:04.841666 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:04.941576 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:04.941449 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:12.524305 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:12.524267 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:12.708817 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:12.708780 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:17.295915 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:17.295874 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:17.392541 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:17.392508 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:19.820096 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:19.819992 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:19.920995 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:19.920951 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:23.259363 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:23.259300 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:23.383423 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:23.383372 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:27.722458 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:27.722414 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" event={"ID":"99231f30-37bc-4ae2-84e6-7ce19cccee07","Type":"ContainerStarted","Data":"31098d48de80ac96b1c4b47ba4e6fb24bded6075e4bac341d3d0e7aa801e5de8"} Jul 21 02:06:27.739115 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:27.739064 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xzm5c" podStartSLOduration=2.33543553 podStartE2EDuration="24.739048165s" totalImagesPullingTime="22.403612635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:06:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:06:27.736436737 +0000 UTC m=+775.553471376" watchObservedRunningTime="2026-07-21 02:06:27.739048165 +0000 UTC m=+775.556082821" Jul 21 02:06:30.318429 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:30.318376 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:30.633755 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:30.633717 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:35.126932 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:35.126903 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:35.230486 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:35.230445 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:37.833806 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:37.833770 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:37.930309 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:37.930276 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:41.320510 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:41.320477 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:41.466670 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:41.466627 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:47.568565 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.568533 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv"] Jul 21 02:06:47.593982 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.593951 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:47.635467 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.635440 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv"] Jul 21 02:06:47.635585 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.635545 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.637679 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.637657 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2drgw\"" Jul 21 02:06:47.756058 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756029 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756180 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756069 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7a7a6da-8efc-44f5-8a68-507dd058475d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756180 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756090 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k922n\" (UniqueName: \"kubernetes.io/projected/a7a7a6da-8efc-44f5-8a68-507dd058475d-kube-api-access-k922n\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756180 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756149 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756324 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756324 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756236 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756324 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756437 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756328 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.756437 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.756351 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857079 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857050 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857230 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857093 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857230 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857230 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857230 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7a7a6da-8efc-44f5-8a68-507dd058475d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857472 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k922n\" (UniqueName: \"kubernetes.io/projected/a7a7a6da-8efc-44f5-8a68-507dd058475d-kube-api-access-k922n\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857472 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857472 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857490 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857545 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857604 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857866 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857659 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.857965 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.857946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7a7a6da-8efc-44f5-8a68-507dd058475d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.859597 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.859579 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.859959 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.859937 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.864798 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.864775 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7a7a6da-8efc-44f5-8a68-507dd058475d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.865273 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.865251 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k922n\" (UniqueName: \"kubernetes.io/projected/a7a7a6da-8efc-44f5-8a68-507dd058475d-kube-api-access-k922n\") pod \"maas-default-gateway-openshift-default-56654dcf7-wsvtv\" (UID: \"a7a7a6da-8efc-44f5-8a68-507dd058475d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:47.946220 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:47.946187 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:48.015891 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.015851 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:48.084330 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.084308 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv"] Jul 21 02:06:48.086121 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:06:48.086096 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a7a6da_8efc_44f5_8a68_507dd058475d.slice/crio-df7a47976646d0c45f6a61231979b1ea3c8e34697ecbbcad29e2045effda06fa WatchSource:0}: Error finding container df7a47976646d0c45f6a61231979b1ea3c8e34697ecbbcad29e2045effda06fa: Status 404 returned error can't find the container with id df7a47976646d0c45f6a61231979b1ea3c8e34697ecbbcad29e2045effda06fa Jul 21 02:06:48.088860 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.088816 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:06:48.088958 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.088892 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:06:48.088958 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.088921 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:06:48.188865 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.188831 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:48.293042 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.293008 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:48.818549 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.818507 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" event={"ID":"a7a7a6da-8efc-44f5-8a68-507dd058475d","Type":"ContainerStarted","Data":"67e61158ec89cf3564109a5fa8c6c430a9df8e483c82e1bb23cc691ea79b39d9"} Jul 21 02:06:48.818928 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.818567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" event={"ID":"a7a7a6da-8efc-44f5-8a68-507dd058475d","Type":"ContainerStarted","Data":"df7a47976646d0c45f6a61231979b1ea3c8e34697ecbbcad29e2045effda06fa"} Jul 21 02:06:48.838183 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.838131 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" podStartSLOduration=1.838115395 podStartE2EDuration="1.838115395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:06:48.836000859 +0000 UTC m=+796.653035497" watchObservedRunningTime="2026-07-21 02:06:48.838115395 +0000 UTC m=+796.655150033" Jul 21 02:06:48.946571 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:48.946545 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:49.842306 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:49.842271 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:49.951337 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:49.951310 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:49.963348 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:49.963293 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:50.075696 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.075665 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:50.291091 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.291017 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:06:50.304314 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.304287 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:06:50.304488 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.304370 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.306552 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.306529 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 21 02:06:50.349699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.349667 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:06:50.381436 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.381391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5972\" (UniqueName: \"kubernetes.io/projected/8872f838-26e7-40b7-8990-157693a9f92a-kube-api-access-x5972\") pod \"limitador-limitador-54d6d988d5-qmm7g\" (UID: \"8872f838-26e7-40b7-8990-157693a9f92a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.381528 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.381493 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8872f838-26e7-40b7-8990-157693a9f92a-config-file\") pod \"limitador-limitador-54d6d988d5-qmm7g\" (UID: \"8872f838-26e7-40b7-8990-157693a9f92a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.482236 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.482212 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5972\" (UniqueName: \"kubernetes.io/projected/8872f838-26e7-40b7-8990-157693a9f92a-kube-api-access-x5972\") pod \"limitador-limitador-54d6d988d5-qmm7g\" (UID: \"8872f838-26e7-40b7-8990-157693a9f92a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.482332 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.482257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8872f838-26e7-40b7-8990-157693a9f92a-config-file\") pod \"limitador-limitador-54d6d988d5-qmm7g\" (UID: \"8872f838-26e7-40b7-8990-157693a9f92a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.482846 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.482830 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8872f838-26e7-40b7-8990-157693a9f92a-config-file\") pod \"limitador-limitador-54d6d988d5-qmm7g\" (UID: \"8872f838-26e7-40b7-8990-157693a9f92a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.490722 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.490696 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5972\" (UniqueName: \"kubernetes.io/projected/8872f838-26e7-40b7-8990-157693a9f92a-kube-api-access-x5972\") pod \"limitador-limitador-54d6d988d5-qmm7g\" (UID: \"8872f838-26e7-40b7-8990-157693a9f92a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.615135 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.615113 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:06:50.727318 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.727282 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:50.741506 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.741486 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:06:50.743775 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:06:50.743752 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8872f838_26e7_40b7_8990_157693a9f92a.slice/crio-7ed40bd610b72782d5e5732fdea8e3a17aa8d04ba310626cbdf7c15897d09df0 WatchSource:0}: Error finding container 7ed40bd610b72782d5e5732fdea8e3a17aa8d04ba310626cbdf7c15897d09df0: Status 404 returned error can't find the container with id 7ed40bd610b72782d5e5732fdea8e3a17aa8d04ba310626cbdf7c15897d09df0 Jul 21 02:06:50.798850 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.798825 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:50.828862 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.828835 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" event={"ID":"8872f838-26e7-40b7-8990-157693a9f92a","Type":"ContainerStarted","Data":"7ed40bd610b72782d5e5732fdea8e3a17aa8d04ba310626cbdf7c15897d09df0"} Jul 21 02:06:50.829015 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.828999 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:50.829821 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:50.829803 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:06:52.932282 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:06:52.932244 2561 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be: Get \"https://s3.us-east-1.amazonaws.com/quayio-production-s3/sha256/fe/feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260721%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260721T020651Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=7efce511ff3817696c9374259aaecf7fae6316b81f5a33dd211c49f5bde744a4\": read tcp 10.0.135.168:49516->16.15.244.107:443: read: connection reset by peer; artifact err: provided artifact is a container image" image="quay.io/kuadrant/limitador:v2.3.0" Jul 21 02:06:52.932725 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:06:52.932341 2561 kuberuntime_manager.go:1459] "Unhandled Error" err="container limitador start failed in pod limitador-limitador-54d6d988d5-qmm7g_kuadrant-system(8872f838-26e7-40b7-8990-157693a9f92a): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be: Get \"https://s3.us-east-1.amazonaws.com/quayio-production-s3/sha256/fe/feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260721%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260721T020651Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=7efce511ff3817696c9374259aaecf7fae6316b81f5a33dd211c49f5bde744a4\": read tcp 10.0.135.168:49516->16.15.244.107:443: read: connection reset by peer; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 21 02:06:52.932725 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:06:52.932379 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"limitador\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be: Get \\\"https://s3.us-east-1.amazonaws.com/quayio-production-s3/sha256/fe/feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260721%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260721T020651Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=7efce511ff3817696c9374259aaecf7fae6316b81f5a33dd211c49f5bde744a4\\\": read tcp 10.0.135.168:49516->16.15.244.107:443: read: connection reset by peer; artifact err: provided artifact is a container image\"" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" podUID="8872f838-26e7-40b7-8990-157693a9f92a" Jul 21 02:06:52.992827 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:52.992796 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:53.091150 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:06:53.091125 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:06:53.845254 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:06:53.845219 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"limitador\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/kuadrant/limitador:v2.3.0\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: reading blob sha256:feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be: Get \\\"https://s3.us-east-1.amazonaws.com/quayio-production-s3/sha256/fe/feef607b73ed06bd1caba835841d5219d05b1973891189ee92996aaf3b69f3be?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260721%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260721T020651Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=7efce511ff3817696c9374259aaecf7fae6316b81f5a33dd211c49f5bde744a4\\\": read tcp 10.0.135.168:49516->16.15.244.107:443: read: connection reset by peer; artifact err: provided artifact is a container image\"" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" podUID="8872f838-26e7-40b7-8990-157693a9f92a" Jul 21 02:07:00.985605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:00.985517 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:01.087033 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:01.086997 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:10.924044 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:10.924006 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" event={"ID":"8872f838-26e7-40b7-8990-157693a9f92a","Type":"ContainerStarted","Data":"cb1212e73aa3f8d38b91f6ee648bd8e7ea64a2806dbd64e6787211c30cf1da87"} Jul 21 02:07:10.924521 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:10.924213 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:07:10.942279 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:10.942229 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" podStartSLOduration=1.8170905830000001 podStartE2EDuration="20.942215486s" totalImagesPullingTime="19.125124903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-21 02:07:10.940307321 +0000 UTC m=+818.757341960" watchObservedRunningTime="2026-07-21 02:07:10.942215486 +0000 UTC m=+818.759250138" Jul 21 02:07:21.930273 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:21.930244 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmm7g" Jul 21 02:07:26.397163 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:26.397125 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:26.495374 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:26.495336 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:26.892512 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:26.892478 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:38.934509 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:38.934464 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:39.306773 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:39.306678 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:45.298360 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:45.298316 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:45.397347 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:45.397311 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:45.449365 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:45.449331 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:48.464943 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:48.464899 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:07:48.566463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:07:48.566399 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:21.669159 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:21.669119 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:21.758356 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:21.758319 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:22.294213 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:22.294183 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:22.566669 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:22.566578 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:22.924592 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:22.924558 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:23.019451 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:23.019419 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:23.220920 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:23.220844 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:08:28.985463 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:28.985422 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:31.294310 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:31.294272 2561 generic.go:356] "Generic (PLEG): container finished" podID="a7a7a6da-8efc-44f5-8a68-507dd058475d" containerID="67e61158ec89cf3564109a5fa8c6c430a9df8e483c82e1bb23cc691ea79b39d9" exitCode=0 Jul 21 02:08:31.294750 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:31.294346 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" event={"ID":"a7a7a6da-8efc-44f5-8a68-507dd058475d","Type":"ContainerDied","Data":"67e61158ec89cf3564109a5fa8c6c430a9df8e483c82e1bb23cc691ea79b39d9"} Jul 21 02:08:31.294750 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:31.294743 2561 scope.go:117] "RemoveContainer" containerID="67e61158ec89cf3564109a5fa8c6c430a9df8e483c82e1bb23cc691ea79b39d9" Jul 21 02:08:31.295716 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:31.295690 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:08:31.295783 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:31.295773 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:08:31.295823 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:31.295803 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 02:08:32.301607 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:32.301572 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" event={"ID":"a7a7a6da-8efc-44f5-8a68-507dd058475d","Type":"ContainerStarted","Data":"a3d3d7d290ef45068d2132858113895daabf95662d1b2d989f36b7c411af853f"} Jul 21 02:08:32.946391 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:32.946345 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:08:32.947709 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:32.947668 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wsvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 21 02:08:32.947709 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:32.947688 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:32.947892 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:32.947733 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" podUID="a7a7a6da-8efc-44f5-8a68-507dd058475d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 21 02:08:33.046862 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:33.046826 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:33.950937 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:33.950905 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wsvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 21 02:08:33.951458 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:33.950966 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" podUID="a7a7a6da-8efc-44f5-8a68-507dd058475d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 21 02:08:33.960159 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:33.960137 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:08:33.962004 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:33.961980 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:08:34.946893 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:34.946853 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wsvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 21 02:08:34.947154 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:34.946915 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" podUID="a7a7a6da-8efc-44f5-8a68-507dd058475d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 21 02:08:35.829882 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:35.829853 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:08:35.947200 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:35.947162 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wsvtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 21 02:08:35.947360 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:35.947231 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" podUID="a7a7a6da-8efc-44f5-8a68-507dd058475d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 21 02:08:36.950921 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:36.950890 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:08:36.951973 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:36.951952 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wsvtv" Jul 21 02:08:39.818737 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:39.818704 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:40.212465 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:40.212428 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:08:45.311305 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:45.311271 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:46.296809 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:46.296761 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:08:50.517386 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:50.517352 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:08:51.704718 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:08:51.704672 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:09:10.324365 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:10.324324 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:09:16.695096 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:16.695051 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:09:16.765590 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:16.765556 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:09:20.526971 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:20.526912 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:09:22.098867 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:22.098836 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:09:31.248944 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:31.248907 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:09:34.192640 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:34.192607 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:09:49.987032 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:49.986999 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw"] Jul 21 02:09:49.989743 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:49.989721 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:49.991922 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:49.991897 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 02:09:49.992482 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:49.992461 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 02:09:49.992602 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:49.992483 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 02:09:49.992602 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:49.992468 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rgs5g\"" Jul 21 02:09:50.002720 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.002698 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw"] Jul 21 02:09:50.029831 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.029805 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:09:50.151099 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.151062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.151240 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.151105 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.151240 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.151144 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.151240 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.151223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.151393 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.151279 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cc606-d598-44ac-af66-3080408d4013-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.151393 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.151348 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszxk\" (UniqueName: \"kubernetes.io/projected/649cc606-d598-44ac-af66-3080408d4013-kube-api-access-kszxk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.251830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.251757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszxk\" (UniqueName: \"kubernetes.io/projected/649cc606-d598-44ac-af66-3080408d4013-kube-api-access-kszxk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.251830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.251823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.251996 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.251840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.251996 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.251868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.251996 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.251906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.251996 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.251928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cc606-d598-44ac-af66-3080408d4013-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.252312 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.252294 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.252364 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.252332 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.252429 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.252380 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.254224 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.254206 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cc606-d598-44ac-af66-3080408d4013-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.254379 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.254362 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cc606-d598-44ac-af66-3080408d4013-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.260972 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.260950 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszxk\" (UniqueName: \"kubernetes.io/projected/649cc606-d598-44ac-af66-3080408d4013-kube-api-access-kszxk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw\" (UID: \"649cc606-d598-44ac-af66-3080408d4013\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.301036 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.301000 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:09:50.437941 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.437916 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw"] Jul 21 02:09:50.438573 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:09:50.438548 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod649cc606_d598_44ac_af66_3080408d4013.slice/crio-3d410598e11954efbef60526ae156d1cd54e6bd07157dcc0786b8900799e29b6 WatchSource:0}: Error finding container 3d410598e11954efbef60526ae156d1cd54e6bd07157dcc0786b8900799e29b6: Status 404 returned error can't find the container with id 3d410598e11954efbef60526ae156d1cd54e6bd07157dcc0786b8900799e29b6 Jul 21 02:09:50.439906 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.439888 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 02:09:50.678458 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:50.678399 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" event={"ID":"649cc606-d598-44ac-af66-3080408d4013","Type":"ContainerStarted","Data":"3d410598e11954efbef60526ae156d1cd54e6bd07157dcc0786b8900799e29b6"} Jul 21 02:09:51.685164 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:51.685085 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" event={"ID":"649cc606-d598-44ac-af66-3080408d4013","Type":"ContainerStarted","Data":"8b9177baa0eef474f6d09136bb6a4089f9e92b3c754564a260122fd1228c50ec"} Jul 21 02:09:51.702474 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:51.702391 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" podStartSLOduration=1.717935955 podStartE2EDuration="2.702370676s" totalImagesPullingTime="984.434721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:09:51.701870181 +0000 UTC m=+979.518904819" watchObservedRunningTime="2026-07-21 02:09:51.702370676 +0000 UTC m=+979.519405313" Jul 21 02:09:52.611284 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:09:52.611247 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:10:00.301478 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:00.301373 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:10:00.301834 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:00.301538 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:10:00.314243 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:00.314222 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:10:00.739364 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:00.739335 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw" Jul 21 02:10:50.698798 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:50.698754 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:10:50.789318 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:50.789280 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:10:51.878785 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:51.878749 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:10:51.989271 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:51.989233 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:10:59.222970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:10:59.222935 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:11:02.528717 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:11:02.528685 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:11:20.682024 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:11:20.681989 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:11:30.784154 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:11:30.784063 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:11:58.304545 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:11:58.304513 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:12:07.310811 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:12:07.310779 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:12:17.803975 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:12:17.803940 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:12:54.630539 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:12:54.630484 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:12:54.732186 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:12:54.732143 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:13:33.992364 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:13:33.992338 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:13:33.994760 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:13:33.993369 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:13:34.247562 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:13:34.247480 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:13:35.445510 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:13:35.445477 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:13:59.391753 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:13:59.391712 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:14:08.506934 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:08.506896 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:14:08.884177 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:08.884145 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:10.585647 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:10.585609 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:15.293068 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:15.293029 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:15.404333 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:15.404295 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:15.607025 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:15.606995 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:15.727253 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:15.727223 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:21.477643 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:21.477600 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:21.569830 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:21.569792 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:14:23.905559 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:23.905527 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:14:33.128087 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:33.128053 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:14:42.203486 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:42.203450 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:14:51.394686 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:14:51.394649 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:15:00.793137 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:15:00.793100 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:15:08.793797 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:15:08.793762 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:15:18.113616 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:15:18.113583 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:15:29.604047 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:15:29.604015 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:15:39.998447 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:15:39.998396 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:15:50.104137 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:15:50.104095 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:16:11.314687 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:16:11.314598 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:16:22.000874 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:16:22.000834 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:16:46.996106 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:16:46.996068 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:16:57.010957 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:16:57.010917 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:17:11.703650 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:11.703615 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:17:23.706983 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:23.706946 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:17:30.234487 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:30.234363 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:17:30.316140 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:30.316106 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:17:33.509325 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:33.509291 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:17:43.401716 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:43.401679 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:17:53.115224 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:17:53.115184 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:18:34.026341 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:18:34.026237 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:18:34.026341 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:18:34.026305 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:19:13.597417 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:19:13.597310 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:19:39.899454 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:19:39.899396 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:19:48.607531 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:19:48.607496 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:20:01.100510 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:01.100475 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:20:12.307004 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:12.306961 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:20:23.099082 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:23.099049 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:20:23.219175 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:23.219142 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:20:34.018930 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:34.018855 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:20:42.845702 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:42.845672 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:20:53.298998 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:20:53.298964 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:21:04.998311 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:04.998273 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:21:13.815268 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:13.815233 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:21:24.206880 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:24.206843 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:21:46.003482 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:46.003450 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:21:55.590751 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.590674 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk"] Jul 21 02:21:55.594308 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.594287 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.596476 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.596453 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-46ee8935-kserve-self-signed-certs\"" Jul 21 02:21:55.610509 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.610487 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk"] Jul 21 02:21:55.653535 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.653509 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:21:55.686158 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.686133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-dshm\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.686273 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.686165 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-model-cache\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.686273 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.686186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxv75\" (UniqueName: \"kubernetes.io/projected/534339da-e3f0-47d6-9609-b6a1edf8a0a8-kube-api-access-nxv75\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.686356 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.686281 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-home\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.686356 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.686311 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tmp-dir\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.686356 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.686325 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tls-certs\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.703803 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.703776 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:21:55.787688 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.787654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-model-cache\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.787688 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.787688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxv75\" (UniqueName: \"kubernetes.io/projected/534339da-e3f0-47d6-9609-b6a1edf8a0a8-kube-api-access-nxv75\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.787893 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.787818 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-home\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.787893 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.787850 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tmp-dir\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.787893 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.787868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tls-certs\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.788069 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.788049 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-model-cache\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.788136 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.788092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-dshm\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.788192 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.788135 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-home\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.788254 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.788204 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tmp-dir\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.790926 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.790905 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-dshm\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.791105 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.791086 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tls-certs\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.795055 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.795037 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxv75\" (UniqueName: \"kubernetes.io/projected/534339da-e3f0-47d6-9609-b6a1edf8a0a8-kube-api-access-nxv75\") pod \"e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:55.903722 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:55.903654 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:21:56.033004 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:56.032981 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk"] Jul 21 02:21:56.034808 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:21:56.034768 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod534339da_e3f0_47d6_9609_b6a1edf8a0a8.slice/crio-20728810a7b2750eabb229e8a3f6a438fc5e8dc5abddb2c19c3d95331c99c064 WatchSource:0}: Error finding container 20728810a7b2750eabb229e8a3f6a438fc5e8dc5abddb2c19c3d95331c99c064: Status 404 returned error can't find the container with id 20728810a7b2750eabb229e8a3f6a438fc5e8dc5abddb2c19c3d95331c99c064 Jul 21 02:21:56.196256 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:56.196177 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" event={"ID":"534339da-e3f0-47d6-9609-b6a1edf8a0a8","Type":"ContainerStarted","Data":"4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da"} Jul 21 02:21:56.196256 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:56.196217 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" event={"ID":"534339da-e3f0-47d6-9609-b6a1edf8a0a8","Type":"ContainerStarted","Data":"20728810a7b2750eabb229e8a3f6a438fc5e8dc5abddb2c19c3d95331c99c064"} Jul 21 02:21:56.214440 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:21:56.214381 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" podStartSLOduration=1.214366266 podStartE2EDuration="1.214366266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:21:56.213332242 +0000 UTC m=+1704.030366877" watchObservedRunningTime="2026-07-21 02:21:56.214366266 +0000 UTC m=+1704.031400904" Jul 21 02:22:01.202934 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.202892 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm"] Jul 21 02:22:01.206790 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.206772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.210071 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.210053 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-46ee8935-kserve-self-signed-certs\"" Jul 21 02:22:01.234474 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.234448 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm"] Jul 21 02:22:01.302575 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.302547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:22:01.339262 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.339237 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwnp\" (UniqueName: \"kubernetes.io/projected/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-kube-api-access-krwnp\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.339395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.339288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-home\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.339395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.339308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-dshm\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.339395 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.339330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tmp-dir\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.339573 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.339548 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-model-cache\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.339627 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.339607 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tls-certs\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441073 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tls-certs\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441241 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441128 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krwnp\" (UniqueName: \"kubernetes.io/projected/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-kube-api-access-krwnp\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441241 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441182 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-home\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441241 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-dshm\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441433 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441239 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tmp-dir\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441433 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-model-cache\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441763 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441736 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-home\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441843 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441771 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-model-cache\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.441843 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.441811 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tmp-dir\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.443350 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.443328 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-dshm\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.443655 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.443635 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tls-certs\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.451180 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.451157 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwnp\" (UniqueName: \"kubernetes.io/projected/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-kube-api-access-krwnp\") pod \"e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.515844 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.515789 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:01.648560 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:01.648531 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm"] Jul 21 02:22:01.651118 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:22:01.651092 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7bbb430_3c72_40f7_a6ca_bb0f2a141dbc.slice/crio-ef1688ad493f14109ae1e3229026815a1d317406920ef9edf6096ace10f20a9c WatchSource:0}: Error finding container ef1688ad493f14109ae1e3229026815a1d317406920ef9edf6096ace10f20a9c: Status 404 returned error can't find the container with id ef1688ad493f14109ae1e3229026815a1d317406920ef9edf6096ace10f20a9c Jul 21 02:22:02.227131 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:02.227096 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" event={"ID":"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc","Type":"ContainerStarted","Data":"4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046"} Jul 21 02:22:02.227131 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:02.227139 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" event={"ID":"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc","Type":"ContainerStarted","Data":"ef1688ad493f14109ae1e3229026815a1d317406920ef9edf6096ace10f20a9c"} Jul 21 02:22:02.245647 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:02.245600 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" podStartSLOduration=1.245584734 podStartE2EDuration="1.245584734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:22:02.244284196 +0000 UTC m=+1710.061318846" watchObservedRunningTime="2026-07-21 02:22:02.245584734 +0000 UTC m=+1710.062619371" Jul 21 02:22:05.904256 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:05.904224 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:22:05.904688 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:05.904308 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:22:05.916900 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:05.916880 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:22:06.260351 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:06.260267 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:22:11.516038 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:11.516005 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:11.516514 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:11.516053 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:11.528700 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:11.528677 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:12.290379 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:12.290349 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:22:14.319988 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:14.319957 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:22:30.612896 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:30.612860 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:22:51.899546 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:51.899515 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk"] Jul 21 02:22:51.924662 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:51.924632 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:22:53.436575 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:53.436541 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm"] Jul 21 02:22:53.461578 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:53.461547 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:22:55.630347 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:22:55.630311 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:23:06.918865 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:06.918808 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" podUID="534339da-e3f0-47d6-9609-b6a1edf8a0a8" containerName="main" containerID="cri-o://4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da" gracePeriod=45 Jul 21 02:23:07.161957 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.161919 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:23:07.243816 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.243736 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-home\") pod \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " Jul 21 02:23:07.243816 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.243784 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-dshm\") pod \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " Jul 21 02:23:07.243816 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.243813 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tmp-dir\") pod \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " Jul 21 02:23:07.244102 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.243846 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tls-certs\") pod \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " Jul 21 02:23:07.244102 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.243881 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-model-cache\") pod \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " Jul 21 02:23:07.244102 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.243915 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxv75\" (UniqueName: \"kubernetes.io/projected/534339da-e3f0-47d6-9609-b6a1edf8a0a8-kube-api-access-nxv75\") pod \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\" (UID: \"534339da-e3f0-47d6-9609-b6a1edf8a0a8\") " Jul 21 02:23:07.244102 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.244035 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-home" (OuterVolumeSpecName: "home") pod "534339da-e3f0-47d6-9609-b6a1edf8a0a8" (UID: "534339da-e3f0-47d6-9609-b6a1edf8a0a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:07.244302 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.244139 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "534339da-e3f0-47d6-9609-b6a1edf8a0a8" (UID: "534339da-e3f0-47d6-9609-b6a1edf8a0a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:07.244302 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.244183 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "534339da-e3f0-47d6-9609-b6a1edf8a0a8" (UID: "534339da-e3f0-47d6-9609-b6a1edf8a0a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:07.244302 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.244253 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tmp-dir\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:07.244302 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.244283 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-home\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:07.246215 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.246181 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "534339da-e3f0-47d6-9609-b6a1edf8a0a8" (UID: "534339da-e3f0-47d6-9609-b6a1edf8a0a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:23:07.246215 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.246192 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534339da-e3f0-47d6-9609-b6a1edf8a0a8-kube-api-access-nxv75" (OuterVolumeSpecName: "kube-api-access-nxv75") pod "534339da-e3f0-47d6-9609-b6a1edf8a0a8" (UID: "534339da-e3f0-47d6-9609-b6a1edf8a0a8"). InnerVolumeSpecName "kube-api-access-nxv75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 02:23:07.246427 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.246236 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-dshm" (OuterVolumeSpecName: "dshm") pod "534339da-e3f0-47d6-9609-b6a1edf8a0a8" (UID: "534339da-e3f0-47d6-9609-b6a1edf8a0a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:07.344849 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.344821 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-dshm\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:07.344849 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.344845 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/534339da-e3f0-47d6-9609-b6a1edf8a0a8-tls-certs\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:07.344849 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.344854 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/534339da-e3f0-47d6-9609-b6a1edf8a0a8-model-cache\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:07.345034 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.344862 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxv75\" (UniqueName: \"kubernetes.io/projected/534339da-e3f0-47d6-9609-b6a1edf8a0a8-kube-api-access-nxv75\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:07.543282 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.543214 2561 generic.go:356] "Generic (PLEG): container finished" podID="534339da-e3f0-47d6-9609-b6a1edf8a0a8" containerID="4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da" exitCode=0 Jul 21 02:23:07.543282 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.543256 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" Jul 21 02:23:07.543282 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.543264 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" event={"ID":"534339da-e3f0-47d6-9609-b6a1edf8a0a8","Type":"ContainerDied","Data":"4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da"} Jul 21 02:23:07.543493 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.543296 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk" event={"ID":"534339da-e3f0-47d6-9609-b6a1edf8a0a8","Type":"ContainerDied","Data":"20728810a7b2750eabb229e8a3f6a438fc5e8dc5abddb2c19c3d95331c99c064"} Jul 21 02:23:07.543493 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.543313 2561 scope.go:117] "RemoveContainer" containerID="4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da" Jul 21 02:23:07.553022 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.553005 2561 scope.go:117] "RemoveContainer" containerID="4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da" Jul 21 02:23:07.553268 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:23:07.553245 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da\": container with ID starting with 4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da not found: ID does not exist" containerID="4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da" Jul 21 02:23:07.553318 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.553279 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da"} err="failed to get container status \"4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da\": rpc error: code = NotFound desc = could not find container \"4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da\": container with ID starting with 4a534a28ec214125f5060db21685080845aa5572f1e41cbf425e21add78b64da not found: ID does not exist" Jul 21 02:23:07.567659 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.567629 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk"] Jul 21 02:23:07.574025 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:07.574005 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-46ee8935-kserve-68596c967-4c4pk"] Jul 21 02:23:08.456378 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.456331 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" podUID="b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" containerName="main" containerID="cri-o://4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046" gracePeriod=45 Jul 21 02:23:08.699304 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.699279 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:23:08.709971 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.709913 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="534339da-e3f0-47d6-9609-b6a1edf8a0a8" path="/var/lib/kubelet/pods/534339da-e3f0-47d6-9609-b6a1edf8a0a8/volumes" Jul 21 02:23:08.757020 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.756994 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krwnp\" (UniqueName: \"kubernetes.io/projected/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-kube-api-access-krwnp\") pod \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " Jul 21 02:23:08.757153 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757028 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tls-certs\") pod \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " Jul 21 02:23:08.757153 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757064 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-dshm\") pod \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " Jul 21 02:23:08.757153 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757109 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-model-cache\") pod \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " Jul 21 02:23:08.757324 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757154 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-home\") pod \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " Jul 21 02:23:08.757324 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757209 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tmp-dir\") pod \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\" (UID: \"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc\") " Jul 21 02:23:08.757440 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757384 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" (UID: "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:08.757515 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757477 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-home" (OuterVolumeSpecName: "home") pod "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" (UID: "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:08.757582 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757557 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" (UID: "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:08.757640 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757620 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-model-cache\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:08.757686 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.757643 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-home\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:08.759221 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.759200 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" (UID: "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 02:23:08.759295 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.759203 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-kube-api-access-krwnp" (OuterVolumeSpecName: "kube-api-access-krwnp") pod "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" (UID: "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc"). InnerVolumeSpecName "kube-api-access-krwnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 02:23:08.759295 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.759254 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-dshm" (OuterVolumeSpecName: "dshm") pod "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" (UID: "b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 02:23:08.858320 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.858280 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tmp-dir\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:08.858320 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.858313 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krwnp\" (UniqueName: \"kubernetes.io/projected/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-kube-api-access-krwnp\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:08.858320 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.858326 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-tls-certs\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:08.858552 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:08.858339 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc-dshm\") on node \"ip-10-0-135-168.ec2.internal\" DevicePath \"\"" Jul 21 02:23:09.555889 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.555856 2561 generic.go:356] "Generic (PLEG): container finished" podID="b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" containerID="4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046" exitCode=0 Jul 21 02:23:09.556289 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.555901 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" Jul 21 02:23:09.556289 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.555933 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" event={"ID":"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc","Type":"ContainerDied","Data":"4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046"} Jul 21 02:23:09.556289 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.555971 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm" event={"ID":"b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc","Type":"ContainerDied","Data":"ef1688ad493f14109ae1e3229026815a1d317406920ef9edf6096ace10f20a9c"} Jul 21 02:23:09.556289 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.555990 2561 scope.go:117] "RemoveContainer" containerID="4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046" Jul 21 02:23:09.565247 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.565229 2561 scope.go:117] "RemoveContainer" containerID="4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046" Jul 21 02:23:09.565527 ip-10-0-135-168 kubenswrapper[2561]: E0721 02:23:09.565511 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046\": container with ID starting with 4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046 not found: ID does not exist" containerID="4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046" Jul 21 02:23:09.565578 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.565537 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046"} err="failed to get container status \"4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046\": rpc error: code = NotFound desc = could not find container \"4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046\": container with ID starting with 4ae64a62d7fdc10b7d32f2d481106739585186d44b334f06dfbcae2606c9e046 not found: ID does not exist" Jul 21 02:23:09.577680 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.577648 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm"] Jul 21 02:23:09.586141 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.586111 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-46ee8935-kserve-bcb67b4cf-94zsm"] Jul 21 02:23:09.824683 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:09.824585 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:23:10.708269 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:10.708237 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" path="/var/lib/kubelet/pods/b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc/volumes" Jul 21 02:23:16.520244 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:16.520204 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:23:31.611699 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:31.611666 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:23:34.057289 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:34.057171 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:23:34.067332 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:34.058194 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qfcwj_9dbab70a-e2fa-4489-b191-4eed783b55a9/ovn-acl-logging/0.log" Jul 21 02:23:40.899779 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:40.899748 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:23:50.799605 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:23:50.799570 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:24:00.913074 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:24:00.913043 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:24:12.311928 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:24:12.311892 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:24:22.018876 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:24:22.018841 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:24:35.221660 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:24:35.221626 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:24:45.943976 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:24:45.943943 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:24:55.310823 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:24:55.310745 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:25:08.417210 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:08.417171 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:25:17.511765 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:17.511727 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:25:24.510546 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:24.510508 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:25:24.752766 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:24.752731 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:25:26.191662 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:26.191620 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:25:39.227466 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:39.227430 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:25:50.826813 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:25:50.826778 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:26:02.007998 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:02.007965 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmm7g"] Jul 21 02:26:11.566421 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:11.566374 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c97f6b6f9-56684_8594d600-acc0-4220-b7d7-c407fe69641d/copy-manifests/0.log" Jul 21 02:26:11.586657 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:11.586617 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c97f6b6f9-56684_8594d600-acc0-4220-b7d7-c407fe69641d/manager/0.log" Jul 21 02:26:14.239607 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:14.239579 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xzm5c_99231f30-37bc-4ae2-84e6-7ce19cccee07/kuadrant-console-plugin/0.log" Jul 21 02:26:14.625165 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:14.625136 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qmm7g_8872f838-26e7-40b7-8990-157693a9f92a/limitador/0.log" Jul 21 02:26:15.113563 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:15.113539 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zzsbn_23eb119c-4f4e-401c-8795-5336619b75c8/istio-proxy/0.log" Jul 21 02:26:15.461995 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:15.461912 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99b8fb9b-j2lbg_66487a6d-99f6-4531-a329-8a818c1d3860/kube-auth-proxy/0.log" Jul 21 02:26:15.596036 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:15.596011 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/1.log" Jul 21 02:26:16.964495 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:16.964468 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-rtrsw_649cc606-d598-44ac-af66-3080408d4013/main/0.log" Jul 21 02:26:18.948844 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:18.948809 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:26:22.803644 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.803611 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tvkpq/must-gather-w245t"] Jul 21 02:26:22.804035 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.803978 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534339da-e3f0-47d6-9609-b6a1edf8a0a8" containerName="main" Jul 21 02:26:22.804035 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.803989 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="534339da-e3f0-47d6-9609-b6a1edf8a0a8" containerName="main" Jul 21 02:26:22.804035 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.804013 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" containerName="main" Jul 21 02:26:22.804035 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.804019 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" containerName="main" Jul 21 02:26:22.804168 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.804084 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7bbb430-3c72-40f7-a6ca-bb0f2a141dbc" containerName="main" Jul 21 02:26:22.804168 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.804097 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="534339da-e3f0-47d6-9609-b6a1edf8a0a8" containerName="main" Jul 21 02:26:22.807353 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.807335 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:22.810161 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.810141 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tvkpq\"/\"kube-root-ca.crt\"" Jul 21 02:26:22.810282 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.810144 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tvkpq\"/\"openshift-service-ca.crt\"" Jul 21 02:26:22.811476 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.811461 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tvkpq\"/\"default-dockercfg-tq2w7\"" Jul 21 02:26:22.824488 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.824467 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:26:22.836128 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.836107 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tvkpq/must-gather-w245t"] Jul 21 02:26:22.889022 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.889000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9f7ca75a-cbfa-4d31-b2d1-57d8be846122-must-gather-output\") pod \"must-gather-w245t\" (UID: \"9f7ca75a-cbfa-4d31-b2d1-57d8be846122\") " pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:22.889124 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.889087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgd8d\" (UniqueName: \"kubernetes.io/projected/9f7ca75a-cbfa-4d31-b2d1-57d8be846122-kube-api-access-pgd8d\") pod \"must-gather-w245t\" (UID: \"9f7ca75a-cbfa-4d31-b2d1-57d8be846122\") " pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:22.925072 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.925050 2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" limit=1024 Jul 21 02:26:22.989970 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.989948 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgd8d\" (UniqueName: \"kubernetes.io/projected/9f7ca75a-cbfa-4d31-b2d1-57d8be846122-kube-api-access-pgd8d\") pod \"must-gather-w245t\" (UID: \"9f7ca75a-cbfa-4d31-b2d1-57d8be846122\") " pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:22.990051 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.990002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9f7ca75a-cbfa-4d31-b2d1-57d8be846122-must-gather-output\") pod \"must-gather-w245t\" (UID: \"9f7ca75a-cbfa-4d31-b2d1-57d8be846122\") " pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:22.990326 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.990310 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9f7ca75a-cbfa-4d31-b2d1-57d8be846122-must-gather-output\") pod \"must-gather-w245t\" (UID: \"9f7ca75a-cbfa-4d31-b2d1-57d8be846122\") " pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:22.998174 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:22.998150 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgd8d\" (UniqueName: \"kubernetes.io/projected/9f7ca75a-cbfa-4d31-b2d1-57d8be846122-kube-api-access-pgd8d\") pod \"must-gather-w245t\" (UID: \"9f7ca75a-cbfa-4d31-b2d1-57d8be846122\") " pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:23.116948 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:23.116927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tvkpq/must-gather-w245t" Jul 21 02:26:23.249115 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:23.249088 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tvkpq/must-gather-w245t"] Jul 21 02:26:23.251207 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:26:23.251181 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f7ca75a_cbfa_4d31_b2d1_57d8be846122.slice/crio-a72e0e2a357dc5471f9cd22067405f5bba4ad9004f0281293789e89d3715930b WatchSource:0}: Error finding container a72e0e2a357dc5471f9cd22067405f5bba4ad9004f0281293789e89d3715930b: Status 404 returned error can't find the container with id a72e0e2a357dc5471f9cd22067405f5bba4ad9004f0281293789e89d3715930b Jul 21 02:26:23.253046 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:23.253028 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 02:26:23.486991 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:23.486909 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tvkpq/must-gather-w245t" event={"ID":"9f7ca75a-cbfa-4d31-b2d1-57d8be846122","Type":"ContainerStarted","Data":"a72e0e2a357dc5471f9cd22067405f5bba4ad9004f0281293789e89d3715930b"} Jul 21 02:26:24.495615 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:24.495567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tvkpq/must-gather-w245t" event={"ID":"9f7ca75a-cbfa-4d31-b2d1-57d8be846122","Type":"ContainerStarted","Data":"c93142b3a0f75e8c06b4911381b3a88196b4632c2c69362f8c17480d539a97d5"} Jul 21 02:26:24.496026 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:24.495629 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tvkpq/must-gather-w245t" event={"ID":"9f7ca75a-cbfa-4d31-b2d1-57d8be846122","Type":"ContainerStarted","Data":"f5190c2dc35becf7f4f665cb675646b7f469e78d92aa25424b70a2695f8bce88"} Jul 21 02:26:24.516119 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:24.516055 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tvkpq/must-gather-w245t" podStartSLOduration=1.860574156 podStartE2EDuration="2.516032865s" totalImagesPullingTime="655.458709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 02:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:26:24.513671947 +0000 UTC m=+1972.330706585" watchObservedRunningTime="2026-07-21 02:26:24.516032865 +0000 UTC m=+1972.333067504" Jul 21 02:26:25.771733 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:25.771657 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wt28r_40ff25a9-41ca-4f8c-aa44-f963d44e5a3b/global-pull-secret-syncer/0.log" Jul 21 02:26:25.827421 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:25.827373 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2pmdj_bde4cbaf-9131-4510-81a3-5d9daf8f801a/konnectivity-agent/0.log" Jul 21 02:26:25.972211 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:25.972177 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-168.ec2.internal_cf74ce26f0de05ee8701a049d7ed8096/haproxy/0.log" Jul 21 02:26:31.377730 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:31.377630 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xzm5c_99231f30-37bc-4ae2-84e6-7ce19cccee07/kuadrant-console-plugin/0.log" Jul 21 02:26:31.584186 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:31.584012 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qmm7g_8872f838-26e7-40b7-8990-157693a9f92a/limitador/0.log" Jul 21 02:26:33.301758 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.301731 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/alertmanager/0.log" Jul 21 02:26:33.349732 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.349704 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/config-reloader/0.log" Jul 21 02:26:33.387024 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.386993 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/kube-rbac-proxy-web/0.log" Jul 21 02:26:33.426891 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.426864 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/kube-rbac-proxy/0.log" Jul 21 02:26:33.471956 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.471922 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/kube-rbac-proxy-metric/0.log" Jul 21 02:26:33.506630 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.506605 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/prom-label-proxy/0.log" Jul 21 02:26:33.540341 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.540297 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d975bb-1a3f-45d8-8b88-2c3df49a8542/init-config-reloader/0.log" Jul 21 02:26:33.582824 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.582791 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-kgjhk_a48756e8-7067-4b95-92cf-e3b7cfde8a8a/cluster-monitoring-operator/0.log" Jul 21 02:26:33.806306 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.806280 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79d4867d56-rh9sl_faef3a87-2cac-4ca7-9249-ed6ad3553847/metrics-server/0.log" Jul 21 02:26:33.845826 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:33.845798 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-cjwhd_97553225-4513-49a4-82c7-dc676e038e86/monitoring-plugin/0.log" Jul 21 02:26:34.003726 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.003692 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s868j_5da262fe-9cc2-40c9-966c-e78f980d7752/node-exporter/0.log" Jul 21 02:26:34.045435 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.045338 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s868j_5da262fe-9cc2-40c9-966c-e78f980d7752/kube-rbac-proxy/0.log" Jul 21 02:26:34.093765 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.093735 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s868j_5da262fe-9cc2-40c9-966c-e78f980d7752/init-textfile/0.log" Jul 21 02:26:34.348589 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.348552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/prometheus/0.log" Jul 21 02:26:34.393335 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.393253 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/config-reloader/0.log" Jul 21 02:26:34.428702 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.428659 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/thanos-sidecar/0.log" Jul 21 02:26:34.458750 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.458711 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/kube-rbac-proxy-web/0.log" Jul 21 02:26:34.485646 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.485608 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/kube-rbac-proxy/0.log" Jul 21 02:26:34.511442 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.511418 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/kube-rbac-proxy-thanos/0.log" Jul 21 02:26:34.555431 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.555388 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cff4d7dc-c42d-4129-a7de-5e456da7465d/init-config-reloader/0.log" Jul 21 02:26:34.587146 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.587112 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-728qf_4e7f2101-c0ad-4340-8cc3-6c7171e801d1/prometheus-operator/0.log" Jul 21 02:26:34.611595 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.611494 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-728qf_4e7f2101-c0ad-4340-8cc3-6c7171e801d1/kube-rbac-proxy/0.log" Jul 21 02:26:34.638522 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.638466 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-8n2vl_472efe22-45c1-46e1-ab1a-2dcfac26ec46/prometheus-operator-admission-webhook/0.log" Jul 21 02:26:34.668489 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.668464 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bc79455f5-4f2ws_55136372-a76c-4e35-8ccc-0159f80db161/telemeter-client/0.log" Jul 21 02:26:34.693044 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.693014 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bc79455f5-4f2ws_55136372-a76c-4e35-8ccc-0159f80db161/reload/0.log" Jul 21 02:26:34.716846 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:34.716822 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bc79455f5-4f2ws_55136372-a76c-4e35-8ccc-0159f80db161/kube-rbac-proxy/0.log" Jul 21 02:26:35.096132 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.096098 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr"] Jul 21 02:26:35.102292 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.102261 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.109279 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.109252 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr"] Jul 21 02:26:35.236283 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.236249 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-lib-modules\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.236491 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.236310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77dsl\" (UniqueName: \"kubernetes.io/projected/cfd60bef-803a-4e75-b794-da87ea8114cd-kube-api-access-77dsl\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.236491 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.236381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-podres\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.236491 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.236471 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-sys\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.236669 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.236510 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-proc\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.337822 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337784 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-podres\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.337979 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-sys\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.337979 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337851 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-proc\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.337979 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-lib-modules\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.337979 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337934 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-sys\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.337979 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337949 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77dsl\" (UniqueName: \"kubernetes.io/projected/cfd60bef-803a-4e75-b794-da87ea8114cd-kube-api-access-77dsl\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.338147 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.337990 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-podres\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.338147 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.338055 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-proc\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.338147 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.338138 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfd60bef-803a-4e75-b794-da87ea8114cd-lib-modules\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.347136 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.347064 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77dsl\" (UniqueName: \"kubernetes.io/projected/cfd60bef-803a-4e75-b794-da87ea8114cd-kube-api-access-77dsl\") pod \"perf-node-gather-daemonset-vmqqr\" (UID: \"cfd60bef-803a-4e75-b794-da87ea8114cd\") " pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.417467 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.417396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:35.599085 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:35.599058 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr"] Jul 21 02:26:35.601870 ip-10-0-135-168 kubenswrapper[2561]: W0721 02:26:35.601827 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcfd60bef_803a_4e75_b794_da87ea8114cd.slice/crio-db60488b0162a2405e38f18e8dcdf925d9b5b7201ca7a180c5617b6ceef54607 WatchSource:0}: Error finding container db60488b0162a2405e38f18e8dcdf925d9b5b7201ca7a180c5617b6ceef54607: Status 404 returned error can't find the container with id db60488b0162a2405e38f18e8dcdf925d9b5b7201ca7a180c5617b6ceef54607 Jul 21 02:26:36.125998 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:36.125972 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-svqv9_d5e054ae-c8cc-41cf-b425-41ed5e1206ff/networking-console-plugin/0.log" Jul 21 02:26:36.581238 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:36.581145 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" event={"ID":"cfd60bef-803a-4e75-b794-da87ea8114cd","Type":"ContainerStarted","Data":"949882c59365133fac031404360d5f250082a689e5e07787610bb661af1a2347"} Jul 21 02:26:36.581238 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:36.581210 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:36.581238 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:36.581234 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" event={"ID":"cfd60bef-803a-4e75-b794-da87ea8114cd","Type":"ContainerStarted","Data":"db60488b0162a2405e38f18e8dcdf925d9b5b7201ca7a180c5617b6ceef54607"} Jul 21 02:26:36.607506 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:36.607452 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" podStartSLOduration=1.607435072 podStartE2EDuration="1.607435072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 02:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 02:26:36.607059145 +0000 UTC m=+1984.424093786" watchObservedRunningTime="2026-07-21 02:26:36.607435072 +0000 UTC m=+1984.424469746" Jul 21 02:26:37.926772 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:37.926723 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qbswg_468a2460-be01-417a-a822-e1664b4c121e/volume-data-source-validator/0.log" Jul 21 02:26:39.216160 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:39.216135 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hq8m_42ad014a-085f-4801-9869-6f03c196609a/dns/0.log" Jul 21 02:26:39.251895 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:39.251864 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hq8m_42ad014a-085f-4801-9869-6f03c196609a/kube-rbac-proxy/0.log" Jul 21 02:26:39.369651 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:39.369621 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lf6nt_67d36da6-d97f-400d-bb0d-2d8986690d0c/dns-node-resolver/0.log" Jul 21 02:26:40.079650 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:40.079610 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d49sq_35bcaf4c-8af4-42cb-8813-ed6fa7f21e7c/node-ca/0.log" Jul 21 02:26:41.087778 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:41.087750 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zzsbn_23eb119c-4f4e-401c-8795-5336619b75c8/istio-proxy/0.log" Jul 21 02:26:41.193885 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:41.193859 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99b8fb9b-j2lbg_66487a6d-99f6-4531-a329-8a818c1d3860/kube-auth-proxy/0.log" Jul 21 02:26:41.226489 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:41.226453 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/0.log" Jul 21 02:26:41.258736 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:41.258705 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/1.log" Jul 21 02:26:42.094251 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:42.094183 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nbst9_e69a0874-21c0-4528-aec5-febde2de9500/serve-healthcheck-canary/0.log" Jul 21 02:26:42.596712 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:42.596686 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tvkpq/perf-node-gather-daemonset-vmqqr" Jul 21 02:26:42.777266 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:42.777207 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n25cz_3e74db41-069c-495d-b709-d950c6ce9c9b/kube-rbac-proxy/0.log" Jul 21 02:26:42.803932 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:42.803902 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n25cz_3e74db41-069c-495d-b709-d950c6ce9c9b/exporter/0.log" Jul 21 02:26:42.834657 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:42.834620 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n25cz_3e74db41-069c-495d-b709-d950c6ce9c9b/extractor/0.log" Jul 21 02:26:45.821278 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:45.821247 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zzsbn_23eb119c-4f4e-401c-8795-5336619b75c8/istio-proxy/0.log" Jul 21 02:26:45.898923 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:45.898887 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99b8fb9b-j2lbg_66487a6d-99f6-4531-a329-8a818c1d3860/kube-auth-proxy/0.log" Jul 21 02:26:45.922667 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:45.922629 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/0.log" Jul 21 02:26:45.953868 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:45.953843 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/1.log" Jul 21 02:26:45.994399 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:45.994360 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c97f6b6f9-56684_8594d600-acc0-4220-b7d7-c407fe69641d/manager/0.log" Jul 21 02:26:46.019652 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:46.019625 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c97f6b6f9-56684_8594d600-acc0-4220-b7d7-c407fe69641d/copy-manifests/0.log" Jul 21 02:26:47.203940 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:47.203906 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zzsbn_23eb119c-4f4e-401c-8795-5336619b75c8/istio-proxy/0.log" Jul 21 02:26:47.291544 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:47.291514 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99b8fb9b-j2lbg_66487a6d-99f6-4531-a329-8a818c1d3860/kube-auth-proxy/0.log" Jul 21 02:26:47.315113 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:47.315067 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/0.log" Jul 21 02:26:47.359300 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:47.359200 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wsvtv_a7a7a6da-8efc-44f5-8a68-507dd058475d/istio-proxy/1.log" Jul 21 02:26:47.510535 ip-10-0-135-168 kubenswrapper[2561]: I0721 02:26:47.510506 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5dfc8579c-tnnbf_baf9a2dc-661e-449d-8f8c-b5bed1085c95/manager/0.log"