Jul 21 20:40:01.043696 ip-10-0-128-226 systemd[1]: Starting Kubernetes Kubelet... Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.400446 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405142 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405154 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405158 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405161 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405165 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405168 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405171 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 20:40:01.489804 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405177 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 20:40:01.469717 ip-10-0-128-226 systemd[1]: Started Kubernetes Kubelet. Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405181 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405186 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405190 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405194 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405197 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405200 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405203 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405207 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405210 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405213 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405215 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405218 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405226 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405229 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405232 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405235 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405238 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405241 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 20:40:01.493592 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405244 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405247 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405250 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405253 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405256 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405259 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405261 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405264 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405267 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405269 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405272 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405275 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405277 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405280 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405283 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405285 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405288 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405291 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405295 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405299 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 20:40:01.494786 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405302 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405305 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405309 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405312 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405316 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405318 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405321 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405324 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405327 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405329 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405332 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405336 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405339 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405342 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405344 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405347 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405350 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405353 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405356 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405359 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 20:40:01.515483 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405361 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405364 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405367 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405369 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405373 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405377 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405379 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405382 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405385 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405387 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405390 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405393 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405395 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405398 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405401 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405403 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405406 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405408 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405411 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405414 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 20:40:01.516143 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405417 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405420 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405422 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405425 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405428 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405443 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405446 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405449 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405452 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405455 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405457 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405461 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.405464 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406156 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406164 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406168 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406171 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406174 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 20:40:01.516737 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406178 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406181 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406183 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406186 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406189 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406192 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406195 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406198 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406201 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406204 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406207 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406209 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406212 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406215 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406217 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406220 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406223 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406227 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406230 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406233 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 20:40:01.517287 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406235 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406238 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406240 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406243 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406246 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406248 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406251 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406253 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406256 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406258 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406262 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406265 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406268 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406271 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406274 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406277 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406279 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406282 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406284 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 20:40:01.517928 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406289 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406293 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406296 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406299 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406302 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406305 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406307 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406310 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406312 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406315 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406317 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406320 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406323 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406326 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406328 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406331 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406334 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406336 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406339 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406342 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 20:40:01.518549 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406344 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406348 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406352 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406355 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406359 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406362 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406364 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406368 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406371 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406373 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406376 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406379 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406382 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406385 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406387 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406390 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406393 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406395 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406398 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406400 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 20:40:01.519094 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406403 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406406 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406408 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406411 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406414 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406417 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406419 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406422 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406425 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406430 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406446 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406449 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406454 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406457 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406460 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406813 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406817 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406820 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406823 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406827 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 20:40:01.519672 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406829 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406834 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406838 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406840 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406843 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406846 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406850 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406854 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406857 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406860 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406863 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406866 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406869 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406872 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406875 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406877 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406880 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406883 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406886 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 20:40:01.520191 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406888 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406891 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406895 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406898 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406901 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406905 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406907 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406910 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406913 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406915 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406919 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406922 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406924 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406927 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406930 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406933 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406936 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406938 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406941 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406944 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 20:40:01.520736 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406946 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406949 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406951 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406954 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406957 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406959 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406962 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406964 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406967 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406969 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406972 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406975 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406977 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406980 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406984 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406987 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.406989 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407000 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407002 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407005 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 20:40:01.521251 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407008 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407010 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407013 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407016 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407018 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407021 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407025 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407029 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407032 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407035 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407037 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407040 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407044 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407047 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407049 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407053 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407056 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407058 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407061 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 20:40:01.521884 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407063 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407066 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407069 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407071 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407074 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407077 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407079 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407083 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407086 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407089 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407093 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407096 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407098 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407101 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407104 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407107 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407177 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407184 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407190 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407194 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407199 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 20:40:01.522430 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407202 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407206 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407215 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407219 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407222 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407225 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407228 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407231 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407234 2576 flags.go:64] FLAG: --cgroup-root="" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407237 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407241 2576 flags.go:64] FLAG: --client-ca-file="" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407243 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407246 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407251 2576 flags.go:64] FLAG: --cluster-domain="" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407254 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407258 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407261 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407264 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407270 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407274 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407277 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407282 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407285 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407288 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 20:40:01.523028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407291 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407294 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407298 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407302 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407305 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407308 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407311 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407314 2576 flags.go:64] FLAG: --enable-server="true" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407318 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407322 2576 flags.go:64] FLAG: --event-burst="100" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407325 2576 flags.go:64] FLAG: --event-qps="50" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407328 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407331 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407334 2576 flags.go:64] FLAG: --eviction-hard="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407338 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407341 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407344 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407347 2576 flags.go:64] FLAG: --eviction-soft="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407350 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407352 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407355 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407358 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407361 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407364 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407367 2576 flags.go:64] FLAG: --feature-gates="" Jul 21 20:40:01.523666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407371 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407376 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407379 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407382 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407386 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407389 2576 flags.go:64] FLAG: --help="false" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407392 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407396 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407399 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407402 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407405 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407408 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407411 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407414 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407417 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407420 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407423 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407427 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407430 2576 flags.go:64] FLAG: --kube-reserved="" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407448 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407452 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407455 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407458 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407461 2576 flags.go:64] FLAG: --lock-file="" Jul 21 20:40:01.524345 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407464 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407467 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407470 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407475 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407478 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407481 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407484 2576 flags.go:64] FLAG: --logging-format="text" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407486 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407490 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407494 2576 flags.go:64] FLAG: --manifest-url="" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407497 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407501 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407506 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407510 2576 flags.go:64] FLAG: --max-pods="110" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407513 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407516 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407519 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407522 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407525 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407528 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407531 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407538 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407541 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407544 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 20:40:01.524987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407547 2576 flags.go:64] FLAG: --pod-cidr="" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407551 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407556 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407559 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407563 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407565 2576 flags.go:64] FLAG: --port="10250" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407568 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407571 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b808670c901feb3" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407575 2576 flags.go:64] FLAG: --qos-reserved="" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407578 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407581 2576 flags.go:64] FLAG: --register-node="true" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407584 2576 flags.go:64] FLAG: --register-with-taints="" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407587 2576 flags.go:64] FLAG: --registry-burst="10" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407591 2576 flags.go:64] FLAG: --registry-qps="5" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407593 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407597 2576 flags.go:64] FLAG: --reserved-memory="" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407600 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407604 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407607 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407610 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407613 2576 flags.go:64] FLAG: --runonce="false" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407617 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407621 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407624 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407627 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 20:40:01.525918 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407630 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407633 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407636 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407639 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407642 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407645 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407648 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407651 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407654 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407657 2576 flags.go:64] FLAG: --system-cgroups="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407660 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407665 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407668 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407671 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407676 2576 flags.go:64] FLAG: --tls-min-version="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407679 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407681 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407684 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407687 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407690 2576 flags.go:64] FLAG: --v="2" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407694 2576 flags.go:64] FLAG: --version="false" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407698 2576 flags.go:64] FLAG: --vmodule="" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407702 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.407705 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407821 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 20:40:01.528905 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407825 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407828 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407831 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407836 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407839 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407842 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407844 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407847 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407850 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407853 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407855 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407860 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407865 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407868 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407871 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407874 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407877 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407880 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407883 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 20:40:01.529854 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407887 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407890 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407893 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407895 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407898 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407901 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407903 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407906 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407909 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407911 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407915 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407919 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407923 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407927 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407930 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407933 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407937 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407940 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407943 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 20:40:01.531976 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407945 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407948 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407951 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407953 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407956 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407959 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407962 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407965 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407967 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407970 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407973 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407976 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407979 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407982 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407984 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407987 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407990 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407993 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407995 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.407998 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408001 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 20:40:01.532711 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408003 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408006 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408009 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408012 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408015 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408018 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408021 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408023 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408028 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408031 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408034 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408036 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408039 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408043 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408047 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408050 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408053 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408055 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408058 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 20:40:01.533845 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408061 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408063 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408066 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408069 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408071 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408081 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408084 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408086 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408089 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408092 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408094 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408097 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408100 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408102 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408105 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408108 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408110 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408115 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408117 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 20:40:01.699297 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.408120 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.408608 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.415943 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.415958 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.415976 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.415986 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416064 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416069 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416074 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416077 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416080 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416083 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416086 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416089 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416093 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 20:40:01.715671 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416096 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416099 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416102 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416110 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416113 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416118 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416121 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416125 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416128 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416131 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416134 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416137 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416140 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416143 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416146 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416149 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416152 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416155 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416158 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416161 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 20:40:01.716320 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416164 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416167 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416170 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416172 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416175 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416178 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416181 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416184 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416186 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416189 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416192 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416194 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416197 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416200 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416203 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416206 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416217 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416223 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416226 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416229 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 20:40:01.719465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416232 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416235 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416238 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416241 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416243 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416246 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416248 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416251 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416254 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416256 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416259 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416262 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416265 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416268 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416271 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416274 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416276 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416279 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416282 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 20:40:01.720042 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416284 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416287 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416290 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416292 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416295 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416298 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416300 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416303 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416306 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416309 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416319 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416321 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416324 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416327 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416329 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416334 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416337 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416341 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416343 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416346 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 20:40:01.720685 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416349 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416352 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416369 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416373 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416376 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416384 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416388 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416391 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416395 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416399 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416402 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.416407 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416595 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416601 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 20:40:01.721231 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416604 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416608 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416610 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416613 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416616 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416619 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416622 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416627 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416629 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416638 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416641 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416643 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416646 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416649 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416651 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416654 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416657 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416659 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416662 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416665 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 20:40:01.721757 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416668 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416670 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416673 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416676 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416680 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416683 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416686 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416690 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416694 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416697 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416700 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416702 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416705 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416708 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416710 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416713 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416716 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416719 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416721 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 20:40:01.722780 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416726 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416729 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416731 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416739 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416742 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416746 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416750 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416753 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416756 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416759 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416762 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416766 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416770 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416773 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416776 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416779 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416782 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416786 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416789 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 20:40:01.723370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416791 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416794 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416797 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416800 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416802 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416805 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416807 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416810 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416813 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416816 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416819 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416822 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416824 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416829 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416831 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416834 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416837 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416845 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416848 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416850 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 20:40:01.914016 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416853 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416856 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416859 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416861 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416864 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416867 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416869 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416872 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416875 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416878 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416882 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416885 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416888 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416890 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416893 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416896 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416899 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416902 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 20:40:01.915245 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:01.416904 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.416909 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.417036 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.419118 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.421267 2576 server.go:1019] "Starting client certificate rotation" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.421375 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.421450 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.425500 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.427464 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.438273 2576 log.go:25] "Validated CRI v1 runtime API" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.440935 2576 log.go:25] "Validated CRI v1 image API" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.442279 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 20:40:01.915902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.444493 2576 fs.go:135] Filesystem UUIDs: map[1816f8de-dab8-41f7-9ecc-dc6fe3319cb7:/dev/nvme0n1p4 2c4e00b4-c725-42bf-801b-d007b466d9d4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 21 20:40:01.916328 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.444512 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.450274 2576 manager.go:216] Machine: {Timestamp:2026-07-21 20:40:01.448295295 +0000 UTC m=+0.302366012 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102610 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a7341da12be98acca0c3353528b6b SystemUUID:ec2a7341-da12-be98-acca-0c3353528b6b BootID:6d2af240-8ff9-4c9f-bf0d-32a4e844a28b Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:53:b8:9d:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:53:b8:9d:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:6a:cb:00:18:3e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.450376 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.450467 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.451102 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.452059 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.452086 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-226.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 20:40:01.916602 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.452776 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.452787 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.452795 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.453401 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.454641 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457457 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457583 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457611 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457623 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457645 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457659 2576 kubelet.go:398] "Adding apiserver pod source" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.457669 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.458018 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.458034 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.459892 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.462583 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.462640 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463766 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463795 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463804 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463812 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463825 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463839 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463851 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463859 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463883 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463946 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463961 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.463970 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.465284 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.465305 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.468797 2576 server.go:1268] "Started kubelet" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.468867 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.468921 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.468979 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.470481 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.470719 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.470998 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.471137 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-226.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 20:40:01.916917 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.472986 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.473736 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.474906 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-226.ec2.internal.18c468a2f61ea6e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-226.ec2.internal,UID:ip-10-0-128-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-226.ec2.internal,},FirstTimestamp:2026-07-21 20:40:01.468753639 +0000 UTC m=+0.322824357,LastTimestamp:2026-07-21 20:40:01.468753639 +0000 UTC m=+0.322824357,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-226.ec2.internal,}" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.477080 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bj6vf" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.478900 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.478978 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.478991 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479662 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479665 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479686 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479778 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479786 2576 factory.go:55] Registering systemd factory Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479802 2576 factory.go:223] Registration of the systemd container factory successfully Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479788 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479985 2576 factory.go:153] Registering CRI-O factory Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.479999 2576 factory.go:223] Registration of the crio container factory successfully Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.480072 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.480096 2576 factory.go:103] Registering Raw factory Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.480111 2576 manager.go:1196] Started watching for new ooms in manager Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.480697 2576 manager.go:349] Starting recovery of all containers Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.481350 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.481462 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.487303 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bj6vf" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.494892 2576 manager.go:354] Recovery completed Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.499200 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.500915 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.500944 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.500958 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.501702 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.501706 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.501719 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.501736 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 21 20:40:01.918255 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.502354 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.502376 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.502416 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.502903 2576 policy_none.go:49] "None policy: Start" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.502918 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.502930 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.503887 2576 policy_none.go:47] "Start" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.550520 2576 manager.go:337] "Starting Device Plugin manager" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.550554 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.550582 2576 server.go:87] "Starting device plugin registration server" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.550824 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.550841 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.550927 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.551006 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.551015 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.551515 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.551563 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.651637 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.652674 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.652713 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.652729 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.652759 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.661586 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.661613 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-226.ec2.internal\": node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.678779 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.700939 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.702160 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.702187 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.702210 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.702244 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.708109 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.779685 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.802867 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal"] Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.802945 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.803986 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.804017 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.804031 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.805419 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.919269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.805530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.805567 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.806149 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.806163 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.806177 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.806184 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.806187 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.806195 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.807385 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.807414 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.808021 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientMemory" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.808041 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.808053 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeHasSufficientPID" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.833971 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-226.ec2.internal\" not found" node="ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.838212 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-226.ec2.internal\" not found" node="ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.880476 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.881731 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/470709dad7cfb08f110d2e7d10c32b8e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-226.ec2.internal\" (UID: \"470709dad7cfb08f110d2e7d10c32b8e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.881754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5213ca01319e306c7ce774b903eb2735-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal\" (UID: \"5213ca01319e306c7ce774b903eb2735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:01.920533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.881774 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5213ca01319e306c7ce774b903eb2735-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal\" (UID: \"5213ca01319e306c7ce774b903eb2735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:01.981527 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.982726 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5213ca01319e306c7ce774b903eb2735-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal\" (UID: \"5213ca01319e306c7ce774b903eb2735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.982752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/470709dad7cfb08f110d2e7d10c32b8e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-226.ec2.internal\" (UID: \"470709dad7cfb08f110d2e7d10c32b8e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.982769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5213ca01319e306c7ce774b903eb2735-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal\" (UID: \"5213ca01319e306c7ce774b903eb2735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.982797 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5213ca01319e306c7ce774b903eb2735-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal\" (UID: \"5213ca01319e306c7ce774b903eb2735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.982832 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5213ca01319e306c7ce774b903eb2735-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal\" (UID: \"5213ca01319e306c7ce774b903eb2735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:01.982841 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/470709dad7cfb08f110d2e7d10c32b8e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-226.ec2.internal\" (UID: \"470709dad7cfb08f110d2e7d10c32b8e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.111786 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.082489 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:02.258877 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.140149 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.258877 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.141255 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.258877 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.183145 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:02.326678 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.283404 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-226.ec2.internal\" not found" Jul 21 20:40:02.327400 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.326851 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.341020 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.381026 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.397340 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.398292 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.413150 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.420552 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.420666 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.420694 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.420733 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.459719 2576 apiserver.go:52] "Watching apiserver" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.474598 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.474963 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-dpg8j","openshift-cluster-node-tuning-operator/tuned-mxht8","openshift-image-registry/node-ca-nflj4","openshift-multus/multus-llsxj","openshift-multus/network-metrics-daemon-gdgtz","openshift-network-operator/iptables-alerter-tlh4q","kube-system/global-pull-secret-syncer-5ttxj","kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7","openshift-dns/node-resolver-pxfnh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal","openshift-multus/multus-additional-cni-plugins-2xmlh","openshift-network-diagnostics/network-check-target-b62zm","openshift-ovn-kubernetes/ovnkube-node-g4gq6"] Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.476574 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.476637 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.477732 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.477792 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.478793 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.479302 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.479913 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.480921 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:02.504391 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.480980 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.481259 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6fqct\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.481337 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.481376 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.481637 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.482139 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.484672 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.485665 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.485842 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486023 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486184 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tqf9d\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486311 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486425 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486589 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q8lpq\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486704 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.486830 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487363 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487412 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487410 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/139d15ea-a86c-4eb5-8cdd-de80d79de8af-dbus\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/286252e9-ce85-4212-9c4f-6dd0f139c56a-serviceca\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487626 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-cni-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0903752f-002d-46ac-8c54-c1cfda27a347-cni-binary-copy\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487673 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-cni-bin\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.505541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-multus-certs\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487729 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vfbs\" (UniqueName: \"kubernetes.io/projected/0903752f-002d-46ac-8c54-c1cfda27a347-kube-api-access-6vfbs\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/139d15ea-a86c-4eb5-8cdd-de80d79de8af-kubelet-config\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-os-release\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487811 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487824 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-cni-multus\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-etc-kubernetes\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487859 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.487871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/286252e9-ce85-4212-9c4f-6dd0f139c56a-host\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488099 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcrk4\" (UniqueName: \"kubernetes.io/projected/286252e9-ce85-4212-9c4f-6dd0f139c56a-kube-api-access-hcrk4\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488129 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-system-cni-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488156 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-k8s-cni-cncf-io\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488429 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-netns\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488507 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-kubelet\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506284 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0903752f-002d-46ac-8c54-c1cfda27a347-multus-daemon-config\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488595 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-cnibin\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488673 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-socket-dir-parent\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488716 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-hostroot\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-conf-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488883 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dj6h5\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.488910 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.491175 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.491213 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mtcdl\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.491705 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.491877 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.491920 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.492252 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.492258 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w5b2x\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.492273 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t9gtd\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.492284 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.492476 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.493353 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.494900 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 20:35:02 +0000 UTC" deadline="2028-04-19 13:18:39.92150425 +0000 UTC" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.494953 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15304h38m37.42655596s" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495414 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495553 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495624 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495753 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 20:40:02.506982 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495770 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 20:40:02.507815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495884 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 20:40:02.507815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.495935 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zcb4g\"" Jul 21 20:40:02.507815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.496131 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 20:40:02.507815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.496133 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-87c2m\"" Jul 21 20:40:02.507815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.498423 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 20:40:02.507815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.500675 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.536196 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f9str" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.548253 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f9str" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.580295 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589633 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-multus-certs\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-slash\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-os-release\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589740 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-multus-certs\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trrn8\" (UniqueName: \"kubernetes.io/projected/76347318-dcc9-4adc-9034-00483679c833-kube-api-access-trrn8\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589805 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-var-lib-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589825 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/139d15ea-a86c-4eb5-8cdd-de80d79de8af-kubelet-config\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589846 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjw99\" (UniqueName: \"kubernetes.io/projected/de325698-eb9b-450c-94e2-b938cfedff1b-kube-api-access-sjw99\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-systemd\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589895 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/139d15ea-a86c-4eb5-8cdd-de80d79de8af-kubelet-config\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-cni-bin\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.728832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-sys-fs\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589969 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-system-cni-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.589987 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a019379e-8329-4f6d-8974-5dfc7da854ec-host-slash\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-sys\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76347318-dcc9-4adc-9034-00483679c833-etc-tuned\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590029 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-system-cni-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590039 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffjh\" (UniqueName: \"kubernetes.io/projected/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-kube-api-access-sffjh\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590066 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gzjf\" (UniqueName: \"kubernetes.io/projected/7967aa89-cc2e-4138-9572-5574be5c03db-kube-api-access-4gzjf\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590129 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-hostroot\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590146 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-conf-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590185 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-hostroot\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590198 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-conf-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590216 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4fbn\" (UniqueName: \"kubernetes.io/projected/90b30272-adf6-4c30-b16a-8a950410acc8-kube-api-access-l4fbn\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:02.729461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovn-node-metrics-cert\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9eb63ef7-b4d9-4cd0-a8f2-68716975488b-konnectivity-ca\") pod \"konnectivity-agent-dpg8j\" (UID: \"9eb63ef7-b4d9-4cd0-a8f2-68716975488b\") " pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/139d15ea-a86c-4eb5-8cdd-de80d79de8af-dbus\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-cni-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590381 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de325698-eb9b-450c-94e2-b938cfedff1b-tmp-dir\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysconfig\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590416 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-kubernetes\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-ovn\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590443 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/139d15ea-a86c-4eb5-8cdd-de80d79de8af-dbus\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-cni-dir\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590483 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-run-ovn-kubernetes\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9eb63ef7-b4d9-4cd0-a8f2-68716975488b-agent-certs\") pod \"konnectivity-agent-dpg8j\" (UID: \"9eb63ef7-b4d9-4cd0-a8f2-68716975488b\") " pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590536 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.730254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/286252e9-ce85-4212-9c4f-6dd0f139c56a-host\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590605 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-kubelet\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590641 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0903752f-002d-46ac-8c54-c1cfda27a347-multus-daemon-config\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590647 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/286252e9-ce85-4212-9c4f-6dd0f139c56a-host\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-node-log\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590679 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-cnibin\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-kubelet\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590702 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-socket-dir-parent\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590720 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysctl-conf\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590736 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-systemd-units\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovnkube-script-lib\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590736 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-cnibin\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590779 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-multus-socket-dir-parent\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590791 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/286252e9-ce85-4212-9c4f-6dd0f139c56a-serviceca\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.730815 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-cni-bin\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-host\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-run-netns\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590890 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-cni-bin\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590925 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cni-binary-copy\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590949 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-socket-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590964 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76347318-dcc9-4adc-9034-00483679c833-tmp\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.590993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591014 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-os-release\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591031 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysctl-d\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-run\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591057 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-os-release\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-var-lib-kubelet\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.591078 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:02.731573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-etc-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-k8s-cni-cncf-io\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-modprobe-d\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-k8s-cni-cncf-io\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.591170 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:40:03.091134606 +0000 UTC m=+1.945205334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-cni-netd\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovnkube-config\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591255 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-device-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591270 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-etc-selinux\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591286 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9qq2\" (UniqueName: \"kubernetes.io/projected/0212f89e-a5b2-445c-97df-8d71dbf1cd76-kube-api-access-p9qq2\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591316 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0903752f-002d-46ac-8c54-c1cfda27a347-cni-binary-copy\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591338 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vfbs\" (UniqueName: \"kubernetes.io/projected/0903752f-002d-46ac-8c54-c1cfda27a347-kube-api-access-6vfbs\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591355 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-log-socket\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cnibin\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591416 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-registration-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591468 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-cni-multus\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591495 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-var-lib-cni-multus\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-etc-kubernetes\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-etc-kubernetes\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591538 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a019379e-8329-4f6d-8974-5dfc7da854ec-iptables-alerter-script\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591560 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8mwd\" (UniqueName: \"kubernetes.io/projected/a019379e-8329-4f6d-8974-5dfc7da854ec-kube-api-access-s8mwd\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-systemd\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591591 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-env-overrides\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591606 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcrk4\" (UniqueName: \"kubernetes.io/projected/286252e9-ce85-4212-9c4f-6dd0f139c56a-kube-api-access-hcrk4\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591642 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-netns\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.732962 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591679 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0903752f-002d-46ac-8c54-c1cfda27a347-host-run-netns\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591753 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de325698-eb9b-450c-94e2-b938cfedff1b-hosts-file\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591772 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-lib-modules\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-kubelet\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.591808 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-system-cni-dir\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.592789 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0903752f-002d-46ac-8c54-c1cfda27a347-multus-daemon-config\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.592934 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/286252e9-ce85-4212-9c4f-6dd0f139c56a-serviceca\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.593538 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0903752f-002d-46ac-8c54-c1cfda27a347-cni-binary-copy\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.609191 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.609209 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.609220 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.609270 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:40:03.109257284 +0000 UTC m=+1.963328005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.617652 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.621294 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vfbs\" (UniqueName: \"kubernetes.io/projected/0903752f-002d-46ac-8c54-c1cfda27a347-kube-api-access-6vfbs\") pod \"multus-llsxj\" (UID: \"0903752f-002d-46ac-8c54-c1cfda27a347\") " pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.621324 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcrk4\" (UniqueName: \"kubernetes.io/projected/286252e9-ce85-4212-9c4f-6dd0f139c56a-kube-api-access-hcrk4\") pod \"node-ca-nflj4\" (UID: \"286252e9-ce85-4212-9c4f-6dd0f139c56a\") " pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.733537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692097 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-systemd\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-env-overrides\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692200 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-systemd\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692257 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de325698-eb9b-450c-94e2-b938cfedff1b-hosts-file\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-lib-modules\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692305 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-kubelet\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692330 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-system-cni-dir\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692350 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de325698-eb9b-450c-94e2-b938cfedff1b-hosts-file\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-slash\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692382 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-os-release\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692406 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trrn8\" (UniqueName: \"kubernetes.io/projected/76347318-dcc9-4adc-9034-00483679c833-kube-api-access-trrn8\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692431 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-var-lib-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692437 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-slash\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692401 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-kubelet\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692455 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-system-cni-dir\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692458 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjw99\" (UniqueName: \"kubernetes.io/projected/de325698-eb9b-450c-94e2-b938cfedff1b-kube-api-access-sjw99\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692511 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692515 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-lib-modules\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-systemd\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692583 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-cni-bin\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.692595 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-sys-fs\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a019379e-8329-4f6d-8974-5dfc7da854ec-host-slash\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-sys\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:02.692717 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:03.19268522 +0000 UTC m=+2.046755929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692722 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-cni-bin\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692738 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76347318-dcc9-4adc-9034-00483679c833-etc-tuned\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.734881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-sys\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffjh\" (UniqueName: \"kubernetes.io/projected/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-kube-api-access-sffjh\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692796 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-sys-fs\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692553 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-os-release\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692807 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a019379e-8329-4f6d-8974-5dfc7da854ec-host-slash\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692776 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-systemd\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692810 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gzjf\" (UniqueName: \"kubernetes.io/projected/7967aa89-cc2e-4138-9572-5574be5c03db-kube-api-access-4gzjf\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692783 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-var-lib-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692854 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-env-overrides\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692868 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4fbn\" (UniqueName: \"kubernetes.io/projected/90b30272-adf6-4c30-b16a-8a950410acc8-kube-api-access-l4fbn\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692894 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovn-node-metrics-cert\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692902 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.692920 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9eb63ef7-b4d9-4cd0-a8f2-68716975488b-konnectivity-ca\") pod \"konnectivity-agent-dpg8j\" (UID: \"9eb63ef7-b4d9-4cd0-a8f2-68716975488b\") " pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de325698-eb9b-450c-94e2-b938cfedff1b-tmp-dir\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.735581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693092 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysconfig\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-kubernetes\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693163 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-ovn\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-run-ovn-kubernetes\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693222 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysconfig\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693229 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9eb63ef7-b4d9-4cd0-a8f2-68716975488b-agent-certs\") pod \"konnectivity-agent-dpg8j\" (UID: \"9eb63ef7-b4d9-4cd0-a8f2-68716975488b\") " pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693264 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693296 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693325 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-node-log\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysctl-conf\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-systemd-units\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de325698-eb9b-450c-94e2-b938cfedff1b-tmp-dir\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-systemd-units\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693432 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-ovn\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693480 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9eb63ef7-b4d9-4cd0-a8f2-68716975488b-konnectivity-ca\") pod \"konnectivity-agent-dpg8j\" (UID: \"9eb63ef7-b4d9-4cd0-a8f2-68716975488b\") " pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.736057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693489 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-run-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovnkube-script-lib\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693541 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693565 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-node-log\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-host\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-run-netns\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693677 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693705 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cni-binary-copy\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693738 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysctl-conf\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-socket-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693788 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-run-netns\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693821 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76347318-dcc9-4adc-9034-00483679c833-tmp\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693849 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693865 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysctl-d\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.881994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693874 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-socket-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693891 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-run\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693905 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-kubernetes\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693918 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-var-lib-kubelet\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693926 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693947 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-etc-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693974 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-modprobe-d\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693986 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.693996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-cni-netd\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694040 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovnkube-config\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-host\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694068 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-device-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694110 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-device-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-etc-selinux\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.882902 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694146 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9qq2\" (UniqueName: \"kubernetes.io/projected/0212f89e-a5b2-445c-97df-8d71dbf1cd76-kube-api-access-p9qq2\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694185 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-log-socket\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694207 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-etc-selinux\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cnibin\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694242 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694302 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694350 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-var-lib-kubelet\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694389 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-etc-openvswitch\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694422 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cni-binary-copy\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694448 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovnkube-script-lib\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694482 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-modprobe-d\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694527 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-run-ovn-kubernetes\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694540 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-host-cni-netd\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694579 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-etc-sysctl-d\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.886026 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694584 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0212f89e-a5b2-445c-97df-8d71dbf1cd76-log-socket\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76347318-dcc9-4adc-9034-00483679c833-run\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694672 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-cnibin\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-registration-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a019379e-8329-4f6d-8974-5dfc7da854ec-iptables-alerter-script\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694912 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8mwd\" (UniqueName: \"kubernetes.io/projected/a019379e-8329-4f6d-8974-5dfc7da854ec-kube-api-access-s8mwd\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.694976 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7967aa89-cc2e-4138-9572-5574be5c03db-registration-dir\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.695215 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovnkube-config\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.695576 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a019379e-8329-4f6d-8974-5dfc7da854ec-iptables-alerter-script\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.695747 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0212f89e-a5b2-445c-97df-8d71dbf1cd76-ovn-node-metrics-cert\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.695798 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76347318-dcc9-4adc-9034-00483679c833-etc-tuned\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.696370 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9eb63ef7-b4d9-4cd0-a8f2-68716975488b-agent-certs\") pod \"konnectivity-agent-dpg8j\" (UID: \"9eb63ef7-b4d9-4cd0-a8f2-68716975488b\") " pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.696704 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76347318-dcc9-4adc-9034-00483679c833-tmp\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.715276 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gzjf\" (UniqueName: \"kubernetes.io/projected/7967aa89-cc2e-4138-9572-5574be5c03db-kube-api-access-4gzjf\") pod \"aws-ebs-csi-driver-node-6brl7\" (UID: \"7967aa89-cc2e-4138-9572-5574be5c03db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:02.886699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.715633 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffjh\" (UniqueName: \"kubernetes.io/projected/4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15-kube-api-access-sffjh\") pod \"multus-additional-cni-plugins-2xmlh\" (UID: \"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15\") " pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.716353 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjw99\" (UniqueName: \"kubernetes.io/projected/de325698-eb9b-450c-94e2-b938cfedff1b-kube-api-access-sjw99\") pod \"node-resolver-pxfnh\" (UID: \"de325698-eb9b-450c-94e2-b938cfedff1b\") " pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.716867 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8mwd\" (UniqueName: \"kubernetes.io/projected/a019379e-8329-4f6d-8974-5dfc7da854ec-kube-api-access-s8mwd\") pod \"iptables-alerter-tlh4q\" (UID: \"a019379e-8329-4f6d-8974-5dfc7da854ec\") " pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.717321 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4fbn\" (UniqueName: \"kubernetes.io/projected/90b30272-adf6-4c30-b16a-8a950410acc8-kube-api-access-l4fbn\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.717748 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trrn8\" (UniqueName: \"kubernetes.io/projected/76347318-dcc9-4adc-9034-00483679c833-kube-api-access-trrn8\") pod \"tuned-mxht8\" (UID: \"76347318-dcc9-4adc-9034-00483679c833\") " pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.722162 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9qq2\" (UniqueName: \"kubernetes.io/projected/0212f89e-a5b2-445c-97df-8d71dbf1cd76-kube-api-access-p9qq2\") pod \"ovnkube-node-g4gq6\" (UID: \"0212f89e-a5b2-445c-97df-8d71dbf1cd76\") " pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.799660 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.807669 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nflj4" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.837492 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-llsxj" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.859933 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tlh4q" Jul 21 20:40:02.887541 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.882889 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.897292 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mxht8" Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.915283 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.935640 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pxfnh" Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.954393 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.962337 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:02.986057 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2bd6b0_afa7_4ac2_8d11_959c15e8ee15.slice/crio-01b33f651875392bf4d45ed7258b1fe866539776f3a99cdaa9b50a851c8391cd WatchSource:0}: Error finding container 01b33f651875392bf4d45ed7258b1fe866539776f3a99cdaa9b50a851c8391cd: Status 404 returned error can't find the container with id 01b33f651875392bf4d45ed7258b1fe866539776f3a99cdaa9b50a851c8391cd Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:02.986327 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470709dad7cfb08f110d2e7d10c32b8e.slice/crio-dd8008eb79b8661d0b36167f62167c5e10f54eeb7039fc6c534427c4879b4496 WatchSource:0}: Error finding container dd8008eb79b8661d0b36167f62167c5e10f54eeb7039fc6c534427c4879b4496: Status 404 returned error can't find the container with id dd8008eb79b8661d0b36167f62167c5e10f54eeb7039fc6c534427c4879b4496 Jul 21 20:40:03.039723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:02.991133 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.053901 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5213ca01319e306c7ce774b903eb2735.slice/crio-7f2b48a87ecb9e8abd8e45af0fbcaf832d189de54938ca64473714b3f994bfe6 WatchSource:0}: Error finding container 7f2b48a87ecb9e8abd8e45af0fbcaf832d189de54938ca64473714b3f994bfe6: Status 404 returned error can't find the container with id 7f2b48a87ecb9e8abd8e45af0fbcaf832d189de54938ca64473714b3f994bfe6 Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.097691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.097821 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.097874 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:40:04.097861014 +0000 UTC m=+2.951931720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.198936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.198975 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.199109 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.199123 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.199133 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.199178 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:40:04.199164617 +0000 UTC m=+3.053235338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.199484 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:03.219228 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.199545 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:04.199534333 +0000 UTC m=+3.053605039 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:03.442285 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.350435 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0903752f_002d_46ac_8c54_c1cfda27a347.slice/crio-2db4e9e22453cc8400ca1db82f205dad298dba4d4ddb3bad5e70f7e590a9deab WatchSource:0}: Error finding container 2db4e9e22453cc8400ca1db82f205dad298dba4d4ddb3bad5e70f7e590a9deab: Status 404 returned error can't find the container with id 2db4e9e22453cc8400ca1db82f205dad298dba4d4ddb3bad5e70f7e590a9deab Jul 21 20:40:03.442285 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.411294 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0212f89e_a5b2_445c_97df_8d71dbf1cd76.slice/crio-20b05e1c696f98d89edc4881b82c5238f2567b9f16134f53e07bf5cffb2b265a WatchSource:0}: Error finding container 20b05e1c696f98d89edc4881b82c5238f2567b9f16134f53e07bf5cffb2b265a: Status 404 returned error can't find the container with id 20b05e1c696f98d89edc4881b82c5238f2567b9f16134f53e07bf5cffb2b265a Jul 21 20:40:03.453416 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.453373 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde325698_eb9b_450c_94e2_b938cfedff1b.slice/crio-9e1ea113550cfd01eb2fdd00bf2c7a5fc63880efe9ed85d13af4338a4c15ca83 WatchSource:0}: Error finding container 9e1ea113550cfd01eb2fdd00bf2c7a5fc63880efe9ed85d13af4338a4c15ca83: Status 404 returned error can't find the container with id 9e1ea113550cfd01eb2fdd00bf2c7a5fc63880efe9ed85d13af4338a4c15ca83 Jul 21 20:40:03.482431 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.482372 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:03.494465 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.494427 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76347318_dcc9_4adc_9034_00483679c833.slice/crio-e6b7ab96c7dea612b9b3d4e2443d657237ea42e74606eaf86c59fdd71efcbfef WatchSource:0}: Error finding container e6b7ab96c7dea612b9b3d4e2443d657237ea42e74606eaf86c59fdd71efcbfef: Status 404 returned error can't find the container with id e6b7ab96c7dea612b9b3d4e2443d657237ea42e74606eaf86c59fdd71efcbfef Jul 21 20:40:03.520448 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.520404 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb63ef7_b4d9_4cd0_a8f2_68716975488b.slice/crio-315fe8c335deaa489425769233b0ebd0f5be5bb32aca765235afb9899e2cad52 WatchSource:0}: Error finding container 315fe8c335deaa489425769233b0ebd0f5be5bb32aca765235afb9899e2cad52: Status 404 returned error can't find the container with id 315fe8c335deaa489425769233b0ebd0f5be5bb32aca765235afb9899e2cad52 Jul 21 20:40:03.550442 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.550390 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 20:35:03 +0000 UTC" deadline="2028-02-08 16:44:39.422946878 +0000 UTC" Jul 21 20:40:03.550565 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.550437 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13604h4m35.872514518s" Jul 21 20:40:03.703923 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.703891 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:03.704095 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.704028 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:03.704542 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.704505 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:03.704668 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.704646 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:03.705190 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.705168 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:03.705296 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:03.705275 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:03.709643 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.709601 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dpg8j" event={"ID":"9eb63ef7-b4d9-4cd0-a8f2-68716975488b","Type":"ContainerStarted","Data":"315fe8c335deaa489425769233b0ebd0f5be5bb32aca765235afb9899e2cad52"} Jul 21 20:40:03.712529 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.712500 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mxht8" event={"ID":"76347318-dcc9-4adc-9034-00483679c833","Type":"ContainerStarted","Data":"e6b7ab96c7dea612b9b3d4e2443d657237ea42e74606eaf86c59fdd71efcbfef"} Jul 21 20:40:03.714570 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.714544 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pxfnh" event={"ID":"de325698-eb9b-450c-94e2-b938cfedff1b","Type":"ContainerStarted","Data":"9e1ea113550cfd01eb2fdd00bf2c7a5fc63880efe9ed85d13af4338a4c15ca83"} Jul 21 20:40:03.717276 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.717252 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"20b05e1c696f98d89edc4881b82c5238f2567b9f16134f53e07bf5cffb2b265a"} Jul 21 20:40:03.721025 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.720997 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-llsxj" event={"ID":"0903752f-002d-46ac-8c54-c1cfda27a347","Type":"ContainerStarted","Data":"2db4e9e22453cc8400ca1db82f205dad298dba4d4ddb3bad5e70f7e590a9deab"} Jul 21 20:40:03.723466 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.723439 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" event={"ID":"5213ca01319e306c7ce774b903eb2735","Type":"ContainerStarted","Data":"7f2b48a87ecb9e8abd8e45af0fbcaf832d189de54938ca64473714b3f994bfe6"} Jul 21 20:40:03.725786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.725763 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" event={"ID":"470709dad7cfb08f110d2e7d10c32b8e","Type":"ContainerStarted","Data":"dd8008eb79b8661d0b36167f62167c5e10f54eeb7039fc6c534427c4879b4496"} Jul 21 20:40:03.729110 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:03.729084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerStarted","Data":"01b33f651875392bf4d45ed7258b1fe866539776f3a99cdaa9b50a851c8391cd"} Jul 21 20:40:03.802743 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:03.802632 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda019379e_8329_4f6d_8974_5dfc7da854ec.slice/crio-792632818c77e99927f98429cfd892b9310399fdbd835dbfdc13bd9cbaa9a910 WatchSource:0}: Error finding container 792632818c77e99927f98429cfd892b9310399fdbd835dbfdc13bd9cbaa9a910: Status 404 returned error can't find the container with id 792632818c77e99927f98429cfd892b9310399fdbd835dbfdc13bd9cbaa9a910 Jul 21 20:40:04.031370 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:04.030826 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286252e9_ce85_4212_9c4f_6dd0f139c56a.slice/crio-56b0fbcb4c1117fa5f9b39ebd8ef60dade4da8e72bf73534d353557d9fd56555 WatchSource:0}: Error finding container 56b0fbcb4c1117fa5f9b39ebd8ef60dade4da8e72bf73534d353557d9fd56555: Status 404 returned error can't find the container with id 56b0fbcb4c1117fa5f9b39ebd8ef60dade4da8e72bf73534d353557d9fd56555 Jul 21 20:40:04.108120 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.108034 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:04.108281 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.108195 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:04.108281 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.108266 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:40:06.108247553 +0000 UTC m=+4.962318263 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.208514 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.208574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.208634 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.208709 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:06.208690128 +0000 UTC m=+5.062760857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.208726 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.208744 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.208756 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:04.208834 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:04.208803 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:40:06.208791007 +0000 UTC m=+5.062861731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:04.365569 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.365467 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:04.551035 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.550991 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 20:35:03 +0000 UTC" deadline="2028-01-26 14:00:06.353901858 +0000 UTC" Jul 21 20:40:04.551035 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.551033 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13289h20m1.802874271s" Jul 21 20:40:04.734175 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.734081 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nflj4" event={"ID":"286252e9-ce85-4212-9c4f-6dd0f139c56a","Type":"ContainerStarted","Data":"56b0fbcb4c1117fa5f9b39ebd8ef60dade4da8e72bf73534d353557d9fd56555"} Jul 21 20:40:04.736627 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.736587 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" event={"ID":"7967aa89-cc2e-4138-9572-5574be5c03db","Type":"ContainerStarted","Data":"c753aa57e800e843186e690cd593fdbb04a713c0d368236045d30a0fae2f11b2"} Jul 21 20:40:04.742467 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:04.742426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tlh4q" event={"ID":"a019379e-8329-4f6d-8974-5dfc7da854ec","Type":"ContainerStarted","Data":"792632818c77e99927f98429cfd892b9310399fdbd835dbfdc13bd9cbaa9a910"} Jul 21 20:40:05.089128 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:05.089043 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 20:40:05.704022 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:05.702975 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:05.704022 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:05.703121 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:05.704022 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:05.703690 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:05.704022 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:05.703786 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:05.704022 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:05.703856 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:05.704022 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:05.703979 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:06.125244 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:06.124477 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:06.125244 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.124655 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:06.125244 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.124723 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:40:10.124704184 +0000 UTC m=+8.978774893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:06.225229 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:06.225331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.225549 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.225616 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:10.225596482 +0000 UTC m=+9.079667191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.226059 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.226081 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.226096 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:06.226179 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:06.226142 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:40:10.22612667 +0000 UTC m=+9.080197393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:07.702805 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:07.702769 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:07.703436 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:07.702923 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:07.703436 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:07.702939 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:07.703436 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:07.703034 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:07.703436 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:07.703165 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:07.703436 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:07.703253 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:09.704424 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:09.703569 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:09.704424 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:09.703719 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:09.704424 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:09.704125 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:09.704424 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:09.704225 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:09.704424 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:09.704300 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:09.704424 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:09.704387 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:10.163909 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:10.163866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:10.164123 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.164030 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:10.164123 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.164108 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:40:18.164090195 +0000 UTC m=+17.018160905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:10.264722 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:10.264580 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:10.264722 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:10.264634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:10.265024 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.264760 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:10.265024 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.264842 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:18.264820689 +0000 UTC m=+17.118891412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:10.265024 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.264861 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:10.265024 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.264884 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:10.265024 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.264895 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:10.265024 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:10.264945 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:40:18.264926764 +0000 UTC m=+17.118997472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:11.703733 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:11.703657 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:11.703733 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:11.703689 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:11.704234 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:11.703767 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:11.704234 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:11.703835 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:11.704234 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:11.703917 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:11.704234 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:11.704013 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:13.702535 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:13.702409 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:13.702535 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:13.702409 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:13.703040 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:13.702559 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:13.703040 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:13.702625 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:13.703040 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:13.702687 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:13.703040 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:13.702792 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:15.705624 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:15.705550 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:15.705624 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:15.705550 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:15.706063 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:15.705660 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:15.706063 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:15.705677 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:15.706063 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:15.705757 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:15.706063 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:15.705831 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:17.703157 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:17.703125 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:17.703601 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:17.703245 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:17.703601 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:17.703250 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:17.703601 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:17.703374 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:17.703601 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:17.703455 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:17.703601 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:17.703530 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:18.228505 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:18.228471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:18.228672 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.228604 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:18.228723 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.228677 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:40:34.228661263 +0000 UTC m=+33.082731973 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:18.329656 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:18.329623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:18.329821 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:18.329680 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:18.329821 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.329788 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:18.329922 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.329837 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:18.329922 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.329860 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:18.329922 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.329873 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:18.329922 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.329871 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:34.329848161 +0000 UTC m=+33.183918867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:18.330056 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:18.329938 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:40:34.329914078 +0000 UTC m=+33.183984809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:19.703351 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:19.703324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:19.703818 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:19.703360 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:19.703818 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:19.703324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:19.703818 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:19.703457 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:19.703818 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:19.703557 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:19.703818 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:19.703654 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:21.704089 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:21.704055 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:21.704559 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:21.704165 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:21.704559 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:21.704212 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:21.704559 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:21.704250 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:21.704559 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:21.704350 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:21.704559 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:21.704470 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:23.703481 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.703308 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:23.703774 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.703311 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:23.703774 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:23.703571 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:23.703774 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:23.703645 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:23.703774 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.703363 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:23.703774 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:23.703731 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:23.795581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.795545 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mxht8" event={"ID":"76347318-dcc9-4adc-9034-00483679c833","Type":"ContainerStarted","Data":"ffc04b93e84bad3d9dde05f273fe6a0ec6ade5fccfa23fbc16e76ecf6b8af263"} Jul 21 20:40:23.797875 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.797847 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-llsxj" event={"ID":"0903752f-002d-46ac-8c54-c1cfda27a347","Type":"ContainerStarted","Data":"24c1ddd33bd96ddde845dfcff7b6d94e3032113c4bbd64fc149e536bebb0ddfe"} Jul 21 20:40:23.799411 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.799380 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" event={"ID":"470709dad7cfb08f110d2e7d10c32b8e","Type":"ContainerStarted","Data":"ba00a38a73144be943298d98cd941fcea30c75658b7d90636bc3e9828877e859"} Jul 21 20:40:23.814885 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.814835 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mxht8" podStartSLOduration=1.769081049 podStartE2EDuration="21.81481687s" totalImagesPullingTime="20.045735821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:23.813995718 +0000 UTC m=+22.668066448" watchObservedRunningTime="2026-07-21 20:40:23.81481687 +0000 UTC m=+22.668887598" Jul 21 20:40:23.833803 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:23.833739 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-llsxj" podStartSLOduration=1.627855482 podStartE2EDuration="21.833720874s" totalImagesPullingTime="20.205865392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:23.832277154 +0000 UTC m=+22.686347892" watchObservedRunningTime="2026-07-21 20:40:23.833720874 +0000 UTC m=+22.687791602" Jul 21 20:40:24.803034 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.802828 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nflj4" event={"ID":"286252e9-ce85-4212-9c4f-6dd0f139c56a","Type":"ContainerStarted","Data":"eb4877843394f6d4831b06f64788c7a66c8439f5e73a680b70fe402d27c7471d"} Jul 21 20:40:24.804274 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.804253 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" event={"ID":"7967aa89-cc2e-4138-9572-5574be5c03db","Type":"ContainerStarted","Data":"2b58d1feb2eab2ac1d264f06ac0b2031c3e8aa72a1a8cd98572fb4591f63afb6"} Jul 21 20:40:24.805614 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.805595 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dpg8j" event={"ID":"9eb63ef7-b4d9-4cd0-a8f2-68716975488b","Type":"ContainerStarted","Data":"d96581800c6848416fe642958b8c657c71e6436e1f169a53f9a7e7d5bb1b1d97"} Jul 21 20:40:24.807032 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.807014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pxfnh" event={"ID":"de325698-eb9b-450c-94e2-b938cfedff1b","Type":"ContainerStarted","Data":"193b1b0422c6539298ff392ac223a0ff8b17918fdd46e1372c91039c7eb5caa6"} Jul 21 20:40:24.808404 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.808384 2576 generic.go:356] "Generic (PLEG): container finished" podID="5213ca01319e306c7ce774b903eb2735" containerID="7f5d51ad82a7a5bacebea0653024d256f29bf85e4bf004c05f558f6a6186eb20" exitCode=0 Jul 21 20:40:24.808499 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.808464 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" event={"ID":"5213ca01319e306c7ce774b903eb2735","Type":"ContainerDied","Data":"7f5d51ad82a7a5bacebea0653024d256f29bf85e4bf004c05f558f6a6186eb20"} Jul 21 20:40:24.811877 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.810957 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15" containerID="04827a1685a355571de0c16f723e125e367af9a83fd00e10c7bd13890369cc4f" exitCode=0 Jul 21 20:40:24.811877 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.811069 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerDied","Data":"04827a1685a355571de0c16f723e125e367af9a83fd00e10c7bd13890369cc4f"} Jul 21 20:40:24.815034 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.815014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"f928caf06d7f1e8c0e4eb39e80e7bd018e649644b55e6163e555af43749574a4"} Jul 21 20:40:24.815113 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.815065 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"c0a106e43662f0f72c37a657e2f94b3a17903c2fb3760dc0290feac51f5d90d4"} Jul 21 20:40:24.815113 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.815097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"8527cf139bed700a1966245b2dcad9bfe86aae930d3c698c3e4a8a8dde4d6664"} Jul 21 20:40:24.815192 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.815123 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"c85e0416ee00322e0ced6cae75d213d982bb52c580b9d2f5aafe8aed55b9aa22"} Jul 21 20:40:24.815192 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.815151 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"e68d1615195defa37d5e08cdb35eed852de09c517ac44c51d6a6ed518d0cf376"} Jul 21 20:40:24.815192 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.815178 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"83283f677257686e601c91ff84b8609277afed30742ce09102a6d8235273230d"} Jul 21 20:40:24.818264 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.818229 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-226.ec2.internal" podStartSLOduration=21.81821231 podStartE2EDuration="21.81821231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:23.848408021 +0000 UTC m=+22.702478750" watchObservedRunningTime="2026-07-21 20:40:24.81821231 +0000 UTC m=+23.672283032" Jul 21 20:40:24.818799 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.818774 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nflj4" podStartSLOduration=3.330118809 podStartE2EDuration="22.818764965s" totalImagesPullingTime="19.488646156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:24.81835534 +0000 UTC m=+23.672426085" watchObservedRunningTime="2026-07-21 20:40:24.818764965 +0000 UTC m=+23.672835718" Jul 21 20:40:24.870466 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.870413 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dpg8j" podStartSLOduration=2.873670179 podStartE2EDuration="22.870401055s" totalImagesPullingTime="19.996730876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:24.844986499 +0000 UTC m=+23.699057244" watchObservedRunningTime="2026-07-21 20:40:24.870401055 +0000 UTC m=+23.724471782" Jul 21 20:40:24.896697 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:24.896500 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pxfnh" podStartSLOduration=7.311944721 podStartE2EDuration="22.896488288s" totalImagesPullingTime="15.584543567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:24.870214687 +0000 UTC m=+23.724285416" watchObservedRunningTime="2026-07-21 20:40:24.896488288 +0000 UTC m=+23.750559015" Jul 21 20:40:25.702512 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.702485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:25.702727 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.702522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:25.702727 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.702489 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:25.702727 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:25.702608 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:25.702895 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:25.702725 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:25.702895 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:25.702815 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:25.769370 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.769235 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 20:40:25.819295 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.819259 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" event={"ID":"7967aa89-cc2e-4138-9572-5574be5c03db","Type":"ContainerStarted","Data":"dc83627068cae4141ad1b80c67e5a8a6a6764b3aec5cdd4ed6a36434320be021"} Jul 21 20:40:25.820930 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.820881 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tlh4q" event={"ID":"a019379e-8329-4f6d-8974-5dfc7da854ec","Type":"ContainerStarted","Data":"8958d075ba9ca8d1cea46fb8114e7dd18795eed2b9d055b27f1416d58867ec70"} Jul 21 20:40:25.837354 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:25.837313 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tlh4q" podStartSLOduration=4.103351188 podStartE2EDuration="23.837298555s" totalImagesPullingTime="19.733947367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:25.836877576 +0000 UTC m=+24.690948322" watchObservedRunningTime="2026-07-21 20:40:25.837298555 +0000 UTC m=+24.691369281" Jul 21 20:40:26.568188 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:26.568082 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T20:40:25.769262493Z","UUID":"d464fc90-0ebe-487d-b026-28f9f7d410a0","Handler":null,"Name":"","Endpoint":""} Jul 21 20:40:26.571688 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:26.571066 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 20:40:26.571688 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:26.571113 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 20:40:26.825504 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:26.825472 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" event={"ID":"5213ca01319e306c7ce774b903eb2735","Type":"ContainerStarted","Data":"37594d26438ff8023133614add50023e7be1a23f610897a2c44cbcd520457843"} Jul 21 20:40:26.843334 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:26.843080 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-226.ec2.internal" podStartSLOduration=23.84306277 podStartE2EDuration="23.84306277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:26.842811577 +0000 UTC m=+25.696882307" watchObservedRunningTime="2026-07-21 20:40:26.84306277 +0000 UTC m=+25.697133497" Jul 21 20:40:27.702974 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:27.702935 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:27.702974 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:27.702973 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:27.703211 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:27.702937 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:27.703211 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:27.703057 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:27.703211 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:27.703127 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:27.703211 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:27.703194 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:27.829196 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:27.829158 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" event={"ID":"7967aa89-cc2e-4138-9572-5574be5c03db","Type":"ContainerStarted","Data":"aeebf855d76d5e07590450b6663d604db5072812658ab30725f297a9ccdcb24b"} Jul 21 20:40:27.832188 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:27.832164 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"4dd3cc2208acc8d02af39ab8dc1711e442cead5126ee75b582a812d9b429a014"} Jul 21 20:40:27.848424 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:27.848386 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6brl7" podStartSLOduration=3.85483969 podStartE2EDuration="25.848375123s" totalImagesPullingTime="21.993535433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:27.847763063 +0000 UTC m=+26.701833791" watchObservedRunningTime="2026-07-21 20:40:27.848375123 +0000 UTC m=+26.702445850" Jul 21 20:40:28.271838 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:28.271811 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:28.272510 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:28.272487 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:28.835029 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:28.834994 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:28.835501 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:28.835479 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dpg8j" Jul 21 20:40:29.703376 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:29.703210 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:29.703523 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:29.703211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:29.703523 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:29.703423 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:29.703628 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:29.703211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:29.703628 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:29.703523 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:29.703688 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:29.703652 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:29.839759 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:29.839732 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" event={"ID":"0212f89e-a5b2-445c-97df-8d71dbf1cd76","Type":"ContainerStarted","Data":"423d34b658668388a7aeb2c3b121dedbd1430797c1f5ad10bb3cbbe4d37a9d00"} Jul 21 20:40:29.868014 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:29.867958 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" podStartSLOduration=7.474145429 podStartE2EDuration="27.867939265s" totalImagesPullingTime="20.393793836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:29.867660746 +0000 UTC m=+28.721731484" watchObservedRunningTime="2026-07-21 20:40:29.867939265 +0000 UTC m=+28.722009992" Jul 21 20:40:30.842991 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.842953 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15" containerID="b949bdf052212014f3f630412f8de9f9e719b2fdf251aa90c3d0cc2c91332d27" exitCode=0 Jul 21 20:40:30.843531 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.843028 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerDied","Data":"b949bdf052212014f3f630412f8de9f9e719b2fdf251aa90c3d0cc2c91332d27"} Jul 21 20:40:30.843882 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.843708 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:30.843882 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.843793 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:30.843882 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.843840 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:30.858038 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.858018 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:30.858249 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:30.858238 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:40:31.698230 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.698001 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b62zm"] Jul 21 20:40:31.698401 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.698265 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:31.698401 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:31.698347 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:31.700712 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.700692 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gdgtz"] Jul 21 20:40:31.700822 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.700809 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:31.700941 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:31.700915 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:31.703937 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.703919 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:31.703999 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:31.703980 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:31.712753 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.712731 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5ttxj"] Jul 21 20:40:31.845025 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:31.845004 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:31.845333 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:31.845078 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:32.849381 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:32.849353 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15" containerID="8466ac3f8adaf3e3c3a9398b2ace0adbe71fa935f763dd7d3c8df06f27fdef58" exitCode=0 Jul 21 20:40:32.849824 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:32.849454 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerDied","Data":"8466ac3f8adaf3e3c3a9398b2ace0adbe71fa935f763dd7d3c8df06f27fdef58"} Jul 21 20:40:33.702724 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:33.702653 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:33.702724 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:33.702671 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:33.702921 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:33.702656 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:33.702921 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:33.702751 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:33.702921 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:33.702818 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:33.702921 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:33.702884 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:34.260568 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:34.260515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:34.260844 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.260611 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:34.260844 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.260655 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret podName:139d15ea-a86c-4eb5-8cdd-de80d79de8af nodeName:}" failed. No retries permitted until 2026-07-21 20:41:06.260643096 +0000 UTC m=+65.114713800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret") pod "global-pull-secret-syncer-5ttxj" (UID: "139d15ea-a86c-4eb5-8cdd-de80d79de8af") : object "kube-system"/"original-pull-secret" not registered Jul 21 20:40:34.295087 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:34.295066 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pxfnh_de325698-eb9b-450c-94e2-b938cfedff1b/dns-node-resolver/0.log" Jul 21 20:40:34.361426 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:34.361403 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:34.361528 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:34.361448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:34.361564 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.361534 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:34.361564 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.361550 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 20:40:34.361564 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.361564 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 20:40:34.361648 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.361573 2576 projected.go:196] Error preparing data for projected volume kube-api-access-dmkg8 for pod openshift-network-diagnostics/network-check-target-b62zm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:34.361648 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.361588 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs podName:90b30272-adf6-4c30-b16a-8a950410acc8 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:06.361572236 +0000 UTC m=+65.215642944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs") pod "network-metrics-daemon-gdgtz" (UID: "90b30272-adf6-4c30-b16a-8a950410acc8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 20:40:34.361648 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:34.361605 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8 podName:6dec3342-39cc-4df8-abd6-ee2ed10cb5da nodeName:}" failed. No retries permitted until 2026-07-21 20:41:06.361595941 +0000 UTC m=+65.215666659 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmkg8" (UniqueName: "kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8") pod "network-check-target-b62zm" (UID: "6dec3342-39cc-4df8-abd6-ee2ed10cb5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 20:40:34.856132 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:34.856106 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15" containerID="5b29b6dd3f789ce4069aeb710232457c0f4d6a9b49f8b95529214cb2c42b7e37" exitCode=0 Jul 21 20:40:34.856230 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:34.856165 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerDied","Data":"5b29b6dd3f789ce4069aeb710232457c0f4d6a9b49f8b95529214cb2c42b7e37"} Jul 21 20:40:35.464383 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:35.464360 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nflj4_286252e9-ce85-4212-9c4f-6dd0f139c56a/node-ca/0.log" Jul 21 20:40:35.703529 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:35.703455 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:35.703529 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:35.703475 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:35.703779 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:35.703585 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5ttxj" podUID="139d15ea-a86c-4eb5-8cdd-de80d79de8af" Jul 21 20:40:35.703779 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:35.703645 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b62zm" podUID="6dec3342-39cc-4df8-abd6-ee2ed10cb5da" Jul 21 20:40:35.703779 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:35.703667 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:35.703972 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:35.703776 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gdgtz" podUID="90b30272-adf6-4c30-b16a-8a950410acc8" Jul 21 20:40:37.427997 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.427929 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-226.ec2.internal" event="NodeReady" Jul 21 20:40:37.428321 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.428032 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 20:40:37.477257 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.477224 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-52g77"] Jul 21 20:40:37.479121 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.479101 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.480291 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.480238 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qfqwd"] Jul 21 20:40:37.481594 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.481570 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 20:40:37.481698 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.481593 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qq4pr\"" Jul 21 20:40:37.481991 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.481975 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 20:40:37.482139 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.482120 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.482487 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.482281 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 20:40:37.484648 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.484494 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 20:40:37.484648 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.484519 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwsp9\"" Jul 21 20:40:37.484648 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.484538 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 20:40:37.484648 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.484548 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 20:40:37.484899 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.484808 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 20:40:37.485089 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.485067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6qkj\" (UniqueName: \"kubernetes.io/projected/ad292553-bfa5-43e2-a1cb-a0ea99d2b93d-kube-api-access-k6qkj\") pod \"ingress-canary-52g77\" (UID: \"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d\") " pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.485176 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.485105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad292553-bfa5-43e2-a1cb-a0ea99d2b93d-cert\") pod \"ingress-canary-52g77\" (UID: \"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d\") " pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.489540 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.489502 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-52g77"] Jul 21 20:40:37.495915 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.495856 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qfqwd"] Jul 21 20:40:37.499562 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.499540 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vqj9x"] Jul 21 20:40:37.501448 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.501422 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.503689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.503670 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 20:40:37.503689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.503697 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 20:40:37.503886 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.503872 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lm6sh\"" Jul 21 20:40:37.512780 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.512761 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqj9x"] Jul 21 20:40:37.585353 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad292553-bfa5-43e2-a1cb-a0ea99d2b93d-cert\") pod \"ingress-canary-52g77\" (UID: \"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d\") " pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.585520 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22cdac8f-b180-4cc2-974d-20aba911febe-config-volume\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.585520 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22cdac8f-b180-4cc2-974d-20aba911febe-tmp-dir\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.585520 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585421 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7854f0d9-93f7-4038-bc67-a38ba102e9e1-data-volume\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.585680 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.585680 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585555 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7854f0d9-93f7-4038-bc67-a38ba102e9e1-crio-socket\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.585680 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.585680 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52cct\" (UniqueName: \"kubernetes.io/projected/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-api-access-52cct\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.585992 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585714 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6qkj\" (UniqueName: \"kubernetes.io/projected/ad292553-bfa5-43e2-a1cb-a0ea99d2b93d-kube-api-access-k6qkj\") pod \"ingress-canary-52g77\" (UID: \"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d\") " pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.585992 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltqwk\" (UniqueName: \"kubernetes.io/projected/22cdac8f-b180-4cc2-974d-20aba911febe-kube-api-access-ltqwk\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.585992 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.585760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.590630 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.590510 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad292553-bfa5-43e2-a1cb-a0ea99d2b93d-cert\") pod \"ingress-canary-52g77\" (UID: \"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d\") " pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.594259 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.594236 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6qkj\" (UniqueName: \"kubernetes.io/projected/ad292553-bfa5-43e2-a1cb-a0ea99d2b93d-kube-api-access-k6qkj\") pod \"ingress-canary-52g77\" (UID: \"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d\") " pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.686021 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.685970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52cct\" (UniqueName: \"kubernetes.io/projected/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-api-access-52cct\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.686021 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltqwk\" (UniqueName: \"kubernetes.io/projected/22cdac8f-b180-4cc2-974d-20aba911febe-kube-api-access-ltqwk\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.686177 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.686177 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22cdac8f-b180-4cc2-974d-20aba911febe-config-volume\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.686177 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:37.686166 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 20:40:37.686329 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:37.686235 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:38.186215646 +0000 UTC m=+37.040286369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : secret "insights-runtime-extractor-tls" not found Jul 21 20:40:37.686329 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686173 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22cdac8f-b180-4cc2-974d-20aba911febe-tmp-dir\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.686329 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7854f0d9-93f7-4038-bc67-a38ba102e9e1-data-volume\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.686516 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.686516 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7854f0d9-93f7-4038-bc67-a38ba102e9e1-crio-socket\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.686516 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686387 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22cdac8f-b180-4cc2-974d-20aba911febe-tmp-dir\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.686516 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.686516 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:37.686479 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 20:40:37.686516 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:37.686497 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 20:40:37.686737 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:37.686527 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls podName:22cdac8f-b180-4cc2-974d-20aba911febe nodeName:}" failed. No retries permitted until 2026-07-21 20:40:38.186511405 +0000 UTC m=+37.040582123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls") pod "dns-default-vqj9x" (UID: "22cdac8f-b180-4cc2-974d-20aba911febe") : secret "dns-default-metrics-tls" not found Jul 21 20:40:37.686737 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:37.686563 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:38.1865475 +0000 UTC m=+37.040618231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : configmap "kube-rbac-proxy" not found Jul 21 20:40:37.686737 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686588 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7854f0d9-93f7-4038-bc67-a38ba102e9e1-data-volume\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.686737 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686604 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22cdac8f-b180-4cc2-974d-20aba911febe-config-volume\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.686737 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.686619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7854f0d9-93f7-4038-bc67-a38ba102e9e1-crio-socket\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.694596 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.694577 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52cct\" (UniqueName: \"kubernetes.io/projected/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-api-access-52cct\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:37.694795 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.694779 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltqwk\" (UniqueName: \"kubernetes.io/projected/22cdac8f-b180-4cc2-974d-20aba911febe-kube-api-access-ltqwk\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:37.702449 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.702424 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:40:37.702566 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.702464 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:40:37.703007 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.702806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:40:37.705288 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.705197 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 20:40:37.705288 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.705199 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdqrl\"" Jul 21 20:40:37.705449 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.705382 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 20:40:37.705512 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.705476 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 20:40:37.705512 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.705496 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 20:40:37.705618 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.705557 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jpthn\"" Jul 21 20:40:37.791675 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.791654 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-52g77" Jul 21 20:40:37.938988 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.938916 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-52g77"] Jul 21 20:40:37.941003 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.940984 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-98q2j"] Jul 21 20:40:37.942664 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:37.942640 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad292553_bfa5_43e2_a1cb_a0ea99d2b93d.slice/crio-17d62473d0e21a1c4910ae667a3cad277354da4a2d5210261695df6798579003 WatchSource:0}: Error finding container 17d62473d0e21a1c4910ae667a3cad277354da4a2d5210261695df6798579003: Status 404 returned error can't find the container with id 17d62473d0e21a1c4910ae667a3cad277354da4a2d5210261695df6798579003 Jul 21 20:40:37.943718 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.943698 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:37.946555 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.946430 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 21 20:40:37.946664 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.946576 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7fgjq\"" Jul 21 20:40:37.946737 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.946690 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 21 20:40:37.947364 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.947176 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 21 20:40:37.947364 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.947213 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 21 20:40:37.954039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.953940 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-98q2j"] Jul 21 20:40:37.988621 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.988598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8a3f712-ddb7-4675-b7bd-148ae21db097-signing-key\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:37.988748 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.988640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l56l\" (UniqueName: \"kubernetes.io/projected/c8a3f712-ddb7-4675-b7bd-148ae21db097-kube-api-access-8l56l\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:37.988748 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:37.988663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8a3f712-ddb7-4675-b7bd-148ae21db097-signing-cabundle\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.089288 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.089249 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8a3f712-ddb7-4675-b7bd-148ae21db097-signing-key\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.089483 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.089303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l56l\" (UniqueName: \"kubernetes.io/projected/c8a3f712-ddb7-4675-b7bd-148ae21db097-kube-api-access-8l56l\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.089483 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.089337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8a3f712-ddb7-4675-b7bd-148ae21db097-signing-cabundle\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.090172 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.090152 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8a3f712-ddb7-4675-b7bd-148ae21db097-signing-cabundle\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.092070 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.092050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8a3f712-ddb7-4675-b7bd-148ae21db097-signing-key\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.099398 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.099377 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l56l\" (UniqueName: \"kubernetes.io/projected/c8a3f712-ddb7-4675-b7bd-148ae21db097-kube-api-access-8l56l\") pod \"service-ca-58598695f7-98q2j\" (UID: \"c8a3f712-ddb7-4675-b7bd-148ae21db097\") " pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.190651 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.190592 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:38.190777 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.190655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:38.190777 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.190685 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:38.190777 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:38.190757 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 20:40:38.190935 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:38.190781 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 20:40:38.190935 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:38.190816 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:39.190801421 +0000 UTC m=+38.044872130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : secret "insights-runtime-extractor-tls" not found Jul 21 20:40:38.190935 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:38.190811 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 20:40:38.190935 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:38.190838 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:39.190822923 +0000 UTC m=+38.044893657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : configmap "kube-rbac-proxy" not found Jul 21 20:40:38.190935 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:38.190855 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls podName:22cdac8f-b180-4cc2-974d-20aba911febe nodeName:}" failed. No retries permitted until 2026-07-21 20:40:39.190846268 +0000 UTC m=+38.044916981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls") pod "dns-default-vqj9x" (UID: "22cdac8f-b180-4cc2-974d-20aba911febe") : secret "dns-default-metrics-tls" not found Jul 21 20:40:38.256171 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.256142 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-98q2j" Jul 21 20:40:38.866152 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:38.866115 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-52g77" event={"ID":"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d","Type":"ContainerStarted","Data":"17d62473d0e21a1c4910ae667a3cad277354da4a2d5210261695df6798579003"} Jul 21 20:40:39.200002 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.199921 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:39.200002 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.199983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:39.200205 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:39.200062 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 20:40:39.200205 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:39.200089 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 20:40:39.200205 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.200063 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:39.200205 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:39.200143 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 20:40:39.200205 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:39.200146 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:41.200128685 +0000 UTC m=+40.054199389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : configmap "kube-rbac-proxy" not found Jul 21 20:40:39.200205 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:39.200205 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls podName:22cdac8f-b180-4cc2-974d-20aba911febe nodeName:}" failed. No retries permitted until 2026-07-21 20:40:41.200189808 +0000 UTC m=+40.054260528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls") pod "dns-default-vqj9x" (UID: "22cdac8f-b180-4cc2-974d-20aba911febe") : secret "dns-default-metrics-tls" not found Jul 21 20:40:39.200465 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:39.200226 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:41.200216376 +0000 UTC m=+40.054287080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : secret "insights-runtime-extractor-tls" not found Jul 21 20:40:39.578072 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.577805 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-98q2j"] Jul 21 20:40:39.583111 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:40:39.583082 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a3f712_ddb7_4675_b7bd_148ae21db097.slice/crio-13aba1b547cdf061d5730e6be7660eaf70d07153f527a88018f312fc016c7b08 WatchSource:0}: Error finding container 13aba1b547cdf061d5730e6be7660eaf70d07153f527a88018f312fc016c7b08: Status 404 returned error can't find the container with id 13aba1b547cdf061d5730e6be7660eaf70d07153f527a88018f312fc016c7b08 Jul 21 20:40:39.872298 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.872197 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15" containerID="866c37e7be6545a38a34f903c2f9ebf836edab77c53ce87736386da72ef11251" exitCode=0 Jul 21 20:40:39.872298 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.872280 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerDied","Data":"866c37e7be6545a38a34f903c2f9ebf836edab77c53ce87736386da72ef11251"} Jul 21 20:40:39.873659 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:39.873633 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-98q2j" event={"ID":"c8a3f712-ddb7-4675-b7bd-148ae21db097","Type":"ContainerStarted","Data":"13aba1b547cdf061d5730e6be7660eaf70d07153f527a88018f312fc016c7b08"} Jul 21 20:40:40.880664 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:40.880560 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15" containerID="f317eeae07f07082cdafff3459b2b1d9889a247d0c1d529e2f4055f9d19b1fff" exitCode=0 Jul 21 20:40:40.880664 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:40.880606 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerDied","Data":"f317eeae07f07082cdafff3459b2b1d9889a247d0c1d529e2f4055f9d19b1fff"} Jul 21 20:40:41.217188 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:41.217155 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:41.217364 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:41.217209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:41.217364 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:41.217273 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:41.217364 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:41.217326 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 20:40:41.217525 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:41.217371 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 20:40:41.217525 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:41.217395 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls podName:22cdac8f-b180-4cc2-974d-20aba911febe nodeName:}" failed. No retries permitted until 2026-07-21 20:40:45.217378114 +0000 UTC m=+44.071448818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls") pod "dns-default-vqj9x" (UID: "22cdac8f-b180-4cc2-974d-20aba911febe") : secret "dns-default-metrics-tls" not found Jul 21 20:40:41.217525 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:41.217404 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 20:40:41.217525 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:41.217426 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:45.217409755 +0000 UTC m=+44.071480460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : configmap "kube-rbac-proxy" not found Jul 21 20:40:41.217525 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:41.217464 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:45.217452925 +0000 UTC m=+44.071523636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : secret "insights-runtime-extractor-tls" not found Jul 21 20:40:42.887850 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:42.887813 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" event={"ID":"4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15","Type":"ContainerStarted","Data":"bef1bc8589d86d047d070ca6be9456069e4c9d29234dcd02baa296f705b0deab"} Jul 21 20:40:42.889308 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:42.889277 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-98q2j" event={"ID":"c8a3f712-ddb7-4675-b7bd-148ae21db097","Type":"ContainerStarted","Data":"5eb0d4074f59a6a7e85c3db0551ca026e7f77d58e9690f80b62078bd5201f4ad"} Jul 21 20:40:42.890590 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:42.890563 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-52g77" event={"ID":"ad292553-bfa5-43e2-a1cb-a0ea99d2b93d","Type":"ContainerStarted","Data":"9fc57ba4dfd93e4dad8da8c1f11af0fc2899c7762fc9046d452085e27ac47b32"} Jul 21 20:40:42.918057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:42.918006 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2xmlh" podStartSLOduration=8.65217905 podStartE2EDuration="40.917989391s" totalImagesPullingTime="32.265810341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:42.916217538 +0000 UTC m=+41.770288275" watchObservedRunningTime="2026-07-21 20:40:42.917989391 +0000 UTC m=+41.772060120" Jul 21 20:40:42.943077 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:42.943039 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-52g77" podStartSLOduration=1.37292595 podStartE2EDuration="5.943025606s" totalImagesPullingTime="4.570099656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:42.941426653 +0000 UTC m=+41.795497380" watchObservedRunningTime="2026-07-21 20:40:42.943025606 +0000 UTC m=+41.797096333" Jul 21 20:40:45.249157 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:45.249122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:45.249179 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:45.249206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:45.249270 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:45.249324 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:45.249277 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:45.249331 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:53.249315128 +0000 UTC m=+52.103385833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : secret "insights-runtime-extractor-tls" not found Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:45.249387 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls podName:22cdac8f-b180-4cc2-974d-20aba911febe nodeName:}" failed. No retries permitted until 2026-07-21 20:40:53.249370308 +0000 UTC m=+52.103441015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls") pod "dns-default-vqj9x" (UID: "22cdac8f-b180-4cc2-974d-20aba911febe") : secret "dns-default-metrics-tls" not found Jul 21 20:40:45.249547 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:45.249400 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:53.249393681 +0000 UTC m=+52.103464386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : configmap "kube-rbac-proxy" not found Jul 21 20:40:53.304203 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.304170 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:53.304661 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.304212 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:53.304661 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.304251 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:53.304661 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:53.304328 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 20:40:53.304661 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:40:53.304383 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls podName:7854f0d9-93f7-4038-bc67-a38ba102e9e1 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:09.304371048 +0000 UTC m=+68.158441753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qfqwd" (UID: "7854f0d9-93f7-4038-bc67-a38ba102e9e1") : secret "insights-runtime-extractor-tls" not found Jul 21 20:40:53.307476 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.307430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22cdac8f-b180-4cc2-974d-20aba911febe-metrics-tls\") pod \"dns-default-vqj9x\" (UID: \"22cdac8f-b180-4cc2-974d-20aba911febe\") " pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:53.319276 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.319247 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7854f0d9-93f7-4038-bc67-a38ba102e9e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:40:53.411004 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.410979 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:53.542786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.542737 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-98q2j" podStartSLOduration=13.60840667 podStartE2EDuration="16.542720324s" totalImagesPullingTime="2.934313654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:42.967424556 +0000 UTC m=+41.821495285" watchObservedRunningTime="2026-07-21 20:40:53.542720324 +0000 UTC m=+52.396791060" Jul 21 20:40:53.542978 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.542964 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqj9x"] Jul 21 20:40:53.921078 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:53.921044 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqj9x" event={"ID":"22cdac8f-b180-4cc2-974d-20aba911febe","Type":"ContainerStarted","Data":"3398176a567f367f69bec2b0f4c3d2d6a2ce97790a9e764d7b834b765b0d1da7"} Jul 21 20:40:55.930696 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:55.930672 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqj9x" event={"ID":"22cdac8f-b180-4cc2-974d-20aba911febe","Type":"ContainerStarted","Data":"b90300a4d0164aad3aca14fe92f3c4c3176237650a4cbca9c2e62989ac252538"} Jul 21 20:40:56.934808 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:56.934773 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqj9x" event={"ID":"22cdac8f-b180-4cc2-974d-20aba911febe","Type":"ContainerStarted","Data":"1fe1903f2ccfeb7b11f8e7cb769bbaa326432a676d50faf097ef31a5d5ba8d3b"} Jul 21 20:40:56.935146 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:56.934895 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vqj9x" Jul 21 20:40:56.952697 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:40:56.952651 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vqj9x" podStartSLOduration=17.799632676 podStartE2EDuration="19.952639295s" totalImagesPullingTime="2.153006619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:56.952340313 +0000 UTC m=+55.806411054" watchObservedRunningTime="2026-07-21 20:40:56.952639295 +0000 UTC m=+55.806710024" Jul 21 20:41:00.330359 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.330327 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx"] Jul 21 20:41:00.332362 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.332339 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.333892 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.333830 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn"] Jul 21 20:41:00.335096 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.335073 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 20:41:00.335267 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.335247 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4pjgr\"" Jul 21 20:41:00.335778 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.335759 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf"] Jul 21 20:41:00.335891 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.335871 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.336634 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.336614 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 20:41:00.337704 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.337688 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.339690 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.339671 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 21 20:41:00.341482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.340709 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 21 20:41:00.341482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.340850 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 21 20:41:00.341482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.340938 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 21 20:41:00.341482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.340861 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8rzs4\"" Jul 21 20:41:00.341482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.341294 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 21 20:41:00.358721 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.358699 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn"] Jul 21 20:41:00.359895 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.359874 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx"] Jul 21 20:41:00.360640 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.360618 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf"] Jul 21 20:41:00.452449 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452417 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-klusterlet-config\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.452559 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/927de61a-16ae-45d1-8b5b-4f03cc1c6af9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-vjqqx\" (UID: \"927de61a-16ae-45d1-8b5b-4f03cc1c6af9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.452559 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452495 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkvpd\" (UniqueName: \"kubernetes.io/projected/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-kube-api-access-gkvpd\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.452559 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452515 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/927de61a-16ae-45d1-8b5b-4f03cc1c6af9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vjqqx\" (UID: \"927de61a-16ae-45d1-8b5b-4f03cc1c6af9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.452675 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-tmp\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.452675 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452611 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qb7j\" (UniqueName: \"kubernetes.io/projected/1e4b5dcc-19af-455f-b45a-7d9c4d2280f9-kube-api-access-7qb7j\") pod \"managed-serviceaccount-addon-agent-7fb6459788-xrsvn\" (UID: \"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.452675 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.452661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e4b5dcc-19af-455f-b45a-7d9c4d2280f9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb6459788-xrsvn\" (UID: \"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.538198 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.538173 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-658585cf49-2lh2j"] Jul 21 20:41:00.540023 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.540008 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.542383 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.542360 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 20:41:00.542658 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.542641 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 20:41:00.542713 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.542689 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9scjn\"" Jul 21 20:41:00.542757 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.542728 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 20:41:00.549336 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.549317 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 20:41:00.553577 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553558 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e4b5dcc-19af-455f-b45a-7d9c4d2280f9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb6459788-xrsvn\" (UID: \"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.553673 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-klusterlet-config\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.553673 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553663 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/927de61a-16ae-45d1-8b5b-4f03cc1c6af9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-vjqqx\" (UID: \"927de61a-16ae-45d1-8b5b-4f03cc1c6af9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.553781 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553688 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkvpd\" (UniqueName: \"kubernetes.io/projected/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-kube-api-access-gkvpd\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.553781 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/927de61a-16ae-45d1-8b5b-4f03cc1c6af9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vjqqx\" (UID: \"927de61a-16ae-45d1-8b5b-4f03cc1c6af9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.553781 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-tmp\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.553922 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.553818 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qb7j\" (UniqueName: \"kubernetes.io/projected/1e4b5dcc-19af-455f-b45a-7d9c4d2280f9-kube-api-access-7qb7j\") pod \"managed-serviceaccount-addon-agent-7fb6459788-xrsvn\" (UID: \"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.554450 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.554417 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/927de61a-16ae-45d1-8b5b-4f03cc1c6af9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-vjqqx\" (UID: \"927de61a-16ae-45d1-8b5b-4f03cc1c6af9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.554797 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.554776 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-tmp\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.555986 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.555964 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/927de61a-16ae-45d1-8b5b-4f03cc1c6af9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-vjqqx\" (UID: \"927de61a-16ae-45d1-8b5b-4f03cc1c6af9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.556065 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.556012 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e4b5dcc-19af-455f-b45a-7d9c4d2280f9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb6459788-xrsvn\" (UID: \"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.556375 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.556361 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-klusterlet-config\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.568196 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.568179 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658585cf49-2lh2j"] Jul 21 20:41:00.569748 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.569720 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qb7j\" (UniqueName: \"kubernetes.io/projected/1e4b5dcc-19af-455f-b45a-7d9c4d2280f9-kube-api-access-7qb7j\") pod \"managed-serviceaccount-addon-agent-7fb6459788-xrsvn\" (UID: \"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.576006 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.575985 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkvpd\" (UniqueName: \"kubernetes.io/projected/2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae-kube-api-access-gkvpd\") pod \"klusterlet-addon-workmgr-9dbc4fd5c-bzssf\" (UID: \"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.644254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.644207 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" Jul 21 20:41:00.654573 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ac132a7b-664d-4d8c-803b-5a0ee6a00543-ca-trust-extracted\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654669 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ac132a7b-664d-4d8c-803b-5a0ee6a00543-image-registry-private-configuration\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654669 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654622 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ac132a7b-664d-4d8c-803b-5a0ee6a00543-trusted-ca\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654669 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-registry-tls\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654669 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ac132a7b-664d-4d8c-803b-5a0ee6a00543-installation-pull-secrets\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654835 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654691 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-bound-sa-token\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654835 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654792 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ac132a7b-664d-4d8c-803b-5a0ee6a00543-registry-certificates\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.654835 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.654815 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7xh4\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-kube-api-access-d7xh4\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.667593 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.667566 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" Jul 21 20:41:00.673905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.673881 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:00.755329 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755274 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ac132a7b-664d-4d8c-803b-5a0ee6a00543-ca-trust-extracted\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755527 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755350 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ac132a7b-664d-4d8c-803b-5a0ee6a00543-image-registry-private-configuration\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755527 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ac132a7b-664d-4d8c-803b-5a0ee6a00543-trusted-ca\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755527 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755455 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-registry-tls\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755527 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755480 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ac132a7b-664d-4d8c-803b-5a0ee6a00543-installation-pull-secrets\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755527 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-bound-sa-token\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755794 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ac132a7b-664d-4d8c-803b-5a0ee6a00543-registry-certificates\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755794 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7xh4\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-kube-api-access-d7xh4\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.755794 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.755671 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ac132a7b-664d-4d8c-803b-5a0ee6a00543-ca-trust-extracted\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.756731 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.756700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ac132a7b-664d-4d8c-803b-5a0ee6a00543-trusted-ca\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.757707 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.757641 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ac132a7b-664d-4d8c-803b-5a0ee6a00543-registry-certificates\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.758365 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.758340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ac132a7b-664d-4d8c-803b-5a0ee6a00543-installation-pull-secrets\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.758850 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.758819 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ac132a7b-664d-4d8c-803b-5a0ee6a00543-image-registry-private-configuration\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.759252 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.759228 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-registry-tls\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.772002 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.771979 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-bound-sa-token\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.772101 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.772037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7xh4\" (UniqueName: \"kubernetes.io/projected/ac132a7b-664d-4d8c-803b-5a0ee6a00543-kube-api-access-d7xh4\") pod \"image-registry-658585cf49-2lh2j\" (UID: \"ac132a7b-664d-4d8c-803b-5a0ee6a00543\") " pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.788040 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.788013 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx"] Jul 21 20:41:00.791337 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:00.791312 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927de61a_16ae_45d1_8b5b_4f03cc1c6af9.slice/crio-1bf96998bda6459d47ff558caced5e7f7679957b5cae494ac2d3c895a88e816a WatchSource:0}: Error finding container 1bf96998bda6459d47ff558caced5e7f7679957b5cae494ac2d3c895a88e816a: Status 404 returned error can't find the container with id 1bf96998bda6459d47ff558caced5e7f7679957b5cae494ac2d3c895a88e816a Jul 21 20:41:00.819163 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.819141 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn"] Jul 21 20:41:00.821829 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:00.821808 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e4b5dcc_19af_455f_b45a_7d9c4d2280f9.slice/crio-fbc0f9c7dd2f45eb7c04a02b691c46c4d9dc0e4de41b4a669d9feb5a3ffd456c WatchSource:0}: Error finding container fbc0f9c7dd2f45eb7c04a02b691c46c4d9dc0e4de41b4a669d9feb5a3ffd456c: Status 404 returned error can't find the container with id fbc0f9c7dd2f45eb7c04a02b691c46c4d9dc0e4de41b4a669d9feb5a3ffd456c Jul 21 20:41:00.839564 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.839544 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf"] Jul 21 20:41:00.844700 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:00.844677 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e04fe01_4784_4dc8_81cd_1ea1c18ad4ae.slice/crio-74eb426bca0a649414e22860da3fd73cdd781a509eb88ec554be62f45f1759d9 WatchSource:0}: Error finding container 74eb426bca0a649414e22860da3fd73cdd781a509eb88ec554be62f45f1759d9: Status 404 returned error can't find the container with id 74eb426bca0a649414e22860da3fd73cdd781a509eb88ec554be62f45f1759d9 Jul 21 20:41:00.849259 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.849231 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:00.946264 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.946231 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" event={"ID":"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9","Type":"ContainerStarted","Data":"fbc0f9c7dd2f45eb7c04a02b691c46c4d9dc0e4de41b4a669d9feb5a3ffd456c"} Jul 21 20:41:00.947398 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.947370 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" event={"ID":"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae","Type":"ContainerStarted","Data":"74eb426bca0a649414e22860da3fd73cdd781a509eb88ec554be62f45f1759d9"} Jul 21 20:41:00.948462 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.948422 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" event={"ID":"927de61a-16ae-45d1-8b5b-4f03cc1c6af9","Type":"ContainerStarted","Data":"1bf96998bda6459d47ff558caced5e7f7679957b5cae494ac2d3c895a88e816a"} Jul 21 20:41:00.989935 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:00.989913 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658585cf49-2lh2j"] Jul 21 20:41:00.992613 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:00.992591 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac132a7b_664d_4d8c_803b_5a0ee6a00543.slice/crio-bbeb258311d58c35637dc8c4678419f0e1aa5cdabf9d810a9540715d4e71a5c7 WatchSource:0}: Error finding container bbeb258311d58c35637dc8c4678419f0e1aa5cdabf9d810a9540715d4e71a5c7: Status 404 returned error can't find the container with id bbeb258311d58c35637dc8c4678419f0e1aa5cdabf9d810a9540715d4e71a5c7 Jul 21 20:41:01.956011 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:01.955152 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" event={"ID":"ac132a7b-664d-4d8c-803b-5a0ee6a00543","Type":"ContainerStarted","Data":"7327172e1de8c789f2b88bbf3e9c261b6224a1a9e94c8aee9d16cf2641531819"} Jul 21 20:41:01.956011 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:01.955214 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" event={"ID":"ac132a7b-664d-4d8c-803b-5a0ee6a00543","Type":"ContainerStarted","Data":"bbeb258311d58c35637dc8c4678419f0e1aa5cdabf9d810a9540715d4e71a5c7"} Jul 21 20:41:01.956011 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:01.955957 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:01.983523 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:01.983467 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" podStartSLOduration=1.983448522 podStartE2EDuration="1.983448522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:01.982381022 +0000 UTC m=+60.836451747" watchObservedRunningTime="2026-07-21 20:41:01.983448522 +0000 UTC m=+60.837519240" Jul 21 20:41:02.870128 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:02.870099 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g4gq6" Jul 21 20:41:05.970056 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.970019 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" event={"ID":"1e4b5dcc-19af-455f-b45a-7d9c4d2280f9","Type":"ContainerStarted","Data":"fcc10a5565329d08b98b747daeba10c9ddbad89f3e2707e6b11c40b4eec46d40"} Jul 21 20:41:05.971455 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.971409 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" event={"ID":"2e04fe01-4784-4dc8-81cd-1ea1c18ad4ae","Type":"ContainerStarted","Data":"0feedebe34061da574077081a70bfb1b9dff6199777c4e89e878c460ac86cdb0"} Jul 21 20:41:05.971558 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.971480 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:05.972875 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.972846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" event={"ID":"927de61a-16ae-45d1-8b5b-4f03cc1c6af9","Type":"ContainerStarted","Data":"3c4976904aff2670f65ba95ab547dd704d141359be5f81ea661fa5d2fb0be8a6"} Jul 21 20:41:05.973316 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.973296 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" Jul 21 20:41:05.986200 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.986163 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb6459788-xrsvn" podStartSLOduration=1.5831448099999998 podStartE2EDuration="5.986151269s" totalImagesPullingTime="4.403006459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:05.985427613 +0000 UTC m=+64.839498351" watchObservedRunningTime="2026-07-21 20:41:05.986151269 +0000 UTC m=+64.840221996" Jul 21 20:41:05.999859 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:05.999823 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-vjqqx" podStartSLOduration=1.578692457 podStartE2EDuration="5.99980737s" totalImagesPullingTime="4.421114913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:05.999782683 +0000 UTC m=+64.853853431" watchObservedRunningTime="2026-07-21 20:41:05.99980737 +0000 UTC m=+64.853878097" Jul 21 20:41:06.016059 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.016019 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-9dbc4fd5c-bzssf" podStartSLOduration=1.62525471 podStartE2EDuration="6.016008941s" totalImagesPullingTime="4.390754231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:06.015263107 +0000 UTC m=+64.869333859" watchObservedRunningTime="2026-07-21 20:41:06.016008941 +0000 UTC m=+64.870079693" Jul 21 20:41:06.307958 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.307888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:41:06.310235 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.310219 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 20:41:06.320146 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.320127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/139d15ea-a86c-4eb5-8cdd-de80d79de8af-original-pull-secret\") pod \"global-pull-secret-syncer-5ttxj\" (UID: \"139d15ea-a86c-4eb5-8cdd-de80d79de8af\") " pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:41:06.408563 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.408537 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:41:06.408656 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.408592 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:41:06.410895 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.410879 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 20:41:06.410968 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.410952 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 20:41:06.420967 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.420947 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b30272-adf6-4c30-b16a-8a950410acc8-metrics-certs\") pod \"network-metrics-daemon-gdgtz\" (UID: \"90b30272-adf6-4c30-b16a-8a950410acc8\") " pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:41:06.421356 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.421335 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 20:41:06.431275 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.431259 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmkg8\" (UniqueName: \"kubernetes.io/projected/6dec3342-39cc-4df8-abd6-ee2ed10cb5da-kube-api-access-dmkg8\") pod \"network-check-target-b62zm\" (UID: \"6dec3342-39cc-4df8-abd6-ee2ed10cb5da\") " pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:41:06.515419 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.515397 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5ttxj" Jul 21 20:41:06.524214 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.524196 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jpthn\"" Jul 21 20:41:06.530216 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.530196 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdqrl\"" Jul 21 20:41:06.533059 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.533044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gdgtz" Jul 21 20:41:06.538750 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.538733 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:41:06.655632 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.655598 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5ttxj"] Jul 21 20:41:06.659326 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:06.659302 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139d15ea_a86c_4eb5_8cdd_de80d79de8af.slice/crio-2a2c1b2bb156ce7611b231d788fe013b846b883c2a1601aabf593668a617229b WatchSource:0}: Error finding container 2a2c1b2bb156ce7611b231d788fe013b846b883c2a1601aabf593668a617229b: Status 404 returned error can't find the container with id 2a2c1b2bb156ce7611b231d788fe013b846b883c2a1601aabf593668a617229b Jul 21 20:41:06.673718 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.673693 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gdgtz"] Jul 21 20:41:06.676904 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:06.676878 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b30272_adf6_4c30_b16a_8a950410acc8.slice/crio-eb0de6edf54e476a21cecf76cd2204a856eabe753d76fd3f13d67da7de1dae25 WatchSource:0}: Error finding container eb0de6edf54e476a21cecf76cd2204a856eabe753d76fd3f13d67da7de1dae25: Status 404 returned error can't find the container with id eb0de6edf54e476a21cecf76cd2204a856eabe753d76fd3f13d67da7de1dae25 Jul 21 20:41:06.693750 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.693730 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b62zm"] Jul 21 20:41:06.696693 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:06.696669 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dec3342_39cc_4df8_abd6_ee2ed10cb5da.slice/crio-e47be6ac0cab493014834159426eb22eb0faf988cf2590725690d87408dcf62f WatchSource:0}: Error finding container e47be6ac0cab493014834159426eb22eb0faf988cf2590725690d87408dcf62f: Status 404 returned error can't find the container with id e47be6ac0cab493014834159426eb22eb0faf988cf2590725690d87408dcf62f Jul 21 20:41:06.939198 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.939136 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vqj9x" Jul 21 20:41:06.977306 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.977274 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b62zm" event={"ID":"6dec3342-39cc-4df8-abd6-ee2ed10cb5da","Type":"ContainerStarted","Data":"e47be6ac0cab493014834159426eb22eb0faf988cf2590725690d87408dcf62f"} Jul 21 20:41:06.978488 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.978462 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5ttxj" event={"ID":"139d15ea-a86c-4eb5-8cdd-de80d79de8af","Type":"ContainerStarted","Data":"2a2c1b2bb156ce7611b231d788fe013b846b883c2a1601aabf593668a617229b"} Jul 21 20:41:06.979482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:06.979458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gdgtz" event={"ID":"90b30272-adf6-4c30-b16a-8a950410acc8","Type":"ContainerStarted","Data":"eb0de6edf54e476a21cecf76cd2204a856eabe753d76fd3f13d67da7de1dae25"} Jul 21 20:41:07.986133 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:07.986101 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gdgtz" event={"ID":"90b30272-adf6-4c30-b16a-8a950410acc8","Type":"ContainerStarted","Data":"720adb5a13b1a01e8da11f6de7075b129ce0cbaab6cbfa11bf54f972b681c71e"} Jul 21 20:41:07.986555 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:07.986153 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gdgtz" event={"ID":"90b30272-adf6-4c30-b16a-8a950410acc8","Type":"ContainerStarted","Data":"3ddd86a2c72500a2bb7b19b94607bed9ab6c06094bf6417ff25a08f4872e00f3"} Jul 21 20:41:08.002759 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:08.002646 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gdgtz" podStartSLOduration=64.940694602 podStartE2EDuration="1m6.002626535s" totalImagesPullingTime="1.061931933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:08.002034272 +0000 UTC m=+66.856105023" watchObservedRunningTime="2026-07-21 20:41:08.002626535 +0000 UTC m=+66.856697264" Jul 21 20:41:09.332114 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:09.332075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:41:09.334998 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:09.334962 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7854f0d9-93f7-4038-bc67-a38ba102e9e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qfqwd\" (UID: \"7854f0d9-93f7-4038-bc67-a38ba102e9e1\") " pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:41:09.601952 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:09.601881 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwsp9\"" Jul 21 20:41:09.609702 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:09.609675 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qfqwd" Jul 21 20:41:11.398097 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:11.398067 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qfqwd"] Jul 21 20:41:11.404304 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:11.404277 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7854f0d9_93f7_4038_bc67_a38ba102e9e1.slice/crio-213c5e744a81d8557bacf811c18c08136b4ebfedd18226403679b928c4386d4b WatchSource:0}: Error finding container 213c5e744a81d8557bacf811c18c08136b4ebfedd18226403679b928c4386d4b: Status 404 returned error can't find the container with id 213c5e744a81d8557bacf811c18c08136b4ebfedd18226403679b928c4386d4b Jul 21 20:41:12.008697 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:12.008660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qfqwd" event={"ID":"7854f0d9-93f7-4038-bc67-a38ba102e9e1","Type":"ContainerStarted","Data":"213c5e744a81d8557bacf811c18c08136b4ebfedd18226403679b928c4386d4b"} Jul 21 20:41:12.010494 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:12.010460 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b62zm" event={"ID":"6dec3342-39cc-4df8-abd6-ee2ed10cb5da","Type":"ContainerStarted","Data":"8fea8c2d9290805b6f3df1dbba8e8cafcc05efee3b6240aae7a7c6a88d8ecb26"} Jul 21 20:41:12.010638 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:12.010608 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:41:12.012248 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:12.012224 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5ttxj" event={"ID":"139d15ea-a86c-4eb5-8cdd-de80d79de8af","Type":"ContainerStarted","Data":"eb9fb640edb883276a0e5cafaaf0cb89750694789ce698cd5d984d35b45ddd69"} Jul 21 20:41:12.028324 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:12.028279 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b62zm" podStartSLOduration=65.456579623 podStartE2EDuration="1m10.028264616s" totalImagesPullingTime="4.571684993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:12.027035649 +0000 UTC m=+70.881106394" watchObservedRunningTime="2026-07-21 20:41:12.028264616 +0000 UTC m=+70.882335463" Jul 21 20:41:12.048574 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:12.048534 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5ttxj" podStartSLOduration=65.433189327 podStartE2EDuration="1m10.048522354s" totalImagesPullingTime="4.615333027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:12.047522513 +0000 UTC m=+70.901593263" watchObservedRunningTime="2026-07-21 20:41:12.048522354 +0000 UTC m=+70.902593075" Jul 21 20:41:17.031644 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:17.031604 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qfqwd" event={"ID":"7854f0d9-93f7-4038-bc67-a38ba102e9e1","Type":"ContainerStarted","Data":"f9aeacb3305f3b49390fd894323d542e4cc6d4b8f8227de6469a89134892e4a4"} Jul 21 20:41:20.044703 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:20.044661 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qfqwd" event={"ID":"7854f0d9-93f7-4038-bc67-a38ba102e9e1","Type":"ContainerStarted","Data":"c6218b257e1a81b60f68a0bde4b43e808e330751d1479e9727ab34d1613a5d57"} Jul 21 20:41:23.056783 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:23.056747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qfqwd" event={"ID":"7854f0d9-93f7-4038-bc67-a38ba102e9e1","Type":"ContainerStarted","Data":"e296044799a157537e86e8719f2972b9ff6387a70ffc9a74d9e6b8fdbc4ee42c"} Jul 21 20:41:23.076397 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:23.076354 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qfqwd" podStartSLOduration=34.673044794 podStartE2EDuration="46.076340454s" totalImagesPullingTime="11.40329566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:23.075399475 +0000 UTC m=+81.929470204" watchObservedRunningTime="2026-07-21 20:41:23.076340454 +0000 UTC m=+81.930411181" Jul 21 20:41:23.965959 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:23.965863 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-658585cf49-2lh2j" Jul 21 20:41:26.902695 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.902659 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp"] Jul 21 20:41:26.907655 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.907631 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:26.910736 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.910716 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gn678\"" Jul 21 20:41:26.911116 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.911085 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 20:41:26.911315 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.911299 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 20:41:26.911676 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.911662 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 20:41:26.911901 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.911671 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 20:41:26.911995 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.911971 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 20:41:26.927393 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.927364 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp"] Jul 21 20:41:26.933797 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.933754 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bchc5"] Jul 21 20:41:26.937425 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.937402 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:26.941595 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.941575 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j5gnj\"" Jul 21 20:41:26.941745 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.941727 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 20:41:26.941802 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.941793 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 20:41:26.942059 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:26.942043 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 20:41:27.062076 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgb4\" (UniqueName: \"kubernetes.io/projected/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-kube-api-access-slgb4\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062206 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062087 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062206 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062125 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-tls\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062206 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bb65\" (UniqueName: \"kubernetes.io/projected/b7cefcae-d372-4dd8-9ced-d12b33cbce11-kube-api-access-4bb65\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.062206 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062179 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-sys\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062221 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-accelerators-collector-config\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-textfile\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-metrics-client-ca\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062313 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062340 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-root\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-wtmp\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.062415 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.062395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7cefcae-d372-4dd8-9ced-d12b33cbce11-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.162660 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-textfile\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.162660 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-metrics-client-ca\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.162660 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-root\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-wtmp\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162745 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-root\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162802 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162808 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-wtmp\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162841 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7cefcae-d372-4dd8-9ced-d12b33cbce11-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.162880 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:41:27.162879 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162889 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgb4\" (UniqueName: \"kubernetes.io/projected/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-kube-api-access-slgb4\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:41:27.162932 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls podName:b7cefcae-d372-4dd8-9ced-d12b33cbce11 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:27.662912763 +0000 UTC m=+86.516983475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-7sqrp" (UID: "b7cefcae-d372-4dd8-9ced-d12b33cbce11") : secret "openshift-state-metrics-tls" not found Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162949 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.162989 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-tls\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163017 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bb65\" (UniqueName: \"kubernetes.io/projected/b7cefcae-d372-4dd8-9ced-d12b33cbce11-kube-api-access-4bb65\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163039 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-sys\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163070 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-accelerators-collector-config\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:41:27.163136 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:41:27.163185 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-tls podName:a1efedd5-d848-44af-bacc-5e2ce0e56a4e nodeName:}" failed. No retries permitted until 2026-07-21 20:41:27.663169736 +0000 UTC m=+86.517240449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-tls") pod "node-exporter-bchc5" (UID: "a1efedd5-d848-44af-bacc-5e2ce0e56a4e") : secret "node-exporter-tls" not found Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163238 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-sys\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163273 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163038 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-textfile\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163846 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163520 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7cefcae-d372-4dd8-9ced-d12b33cbce11-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.163846 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163640 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-accelerators-collector-config\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.163950 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.163844 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-metrics-client-ca\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.165576 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.165556 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.165689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.165611 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.172029 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.172006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bb65\" (UniqueName: \"kubernetes.io/projected/b7cefcae-d372-4dd8-9ced-d12b33cbce11-kube-api-access-4bb65\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.173351 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.173331 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgb4\" (UniqueName: \"kubernetes.io/projected/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-kube-api-access-slgb4\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.666515 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.666486 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:27.666671 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.666530 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-tls\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.666671 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:41:27.666639 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 20:41:27.666776 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:41:27.666717 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls podName:b7cefcae-d372-4dd8-9ced-d12b33cbce11 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:28.66669905 +0000 UTC m=+87.520769770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-7sqrp" (UID: "b7cefcae-d372-4dd8-9ced-d12b33cbce11") : secret "openshift-state-metrics-tls" not found Jul 21 20:41:27.668645 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.668623 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a1efedd5-d848-44af-bacc-5e2ce0e56a4e-node-exporter-tls\") pod \"node-exporter-bchc5\" (UID: \"a1efedd5-d848-44af-bacc-5e2ce0e56a4e\") " pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.854289 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.854263 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bchc5" Jul 21 20:41:27.868645 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:27.868621 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1efedd5_d848_44af_bacc_5e2ce0e56a4e.slice/crio-e4576d9a1583dbc8fc27543369b013774722a6c2f9ef12830192cb755c37709c WatchSource:0}: Error finding container e4576d9a1583dbc8fc27543369b013774722a6c2f9ef12830192cb755c37709c: Status 404 returned error can't find the container with id e4576d9a1583dbc8fc27543369b013774722a6c2f9ef12830192cb755c37709c Jul 21 20:41:27.979105 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.979083 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 20:41:27.984910 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.984893 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:27.987709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987314 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 20:41:27.987709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987319 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 20:41:27.987709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 20:41:27.987709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987536 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 20:41:27.987709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987574 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 20:41:27.987709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987651 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 20:41:27.988080 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.987856 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 20:41:27.988454 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.988373 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 20:41:27.988454 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.988422 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9vcm5\"" Jul 21 20:41:27.988859 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:27.988448 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 20:41:28.003840 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.003816 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 20:41:28.076481 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.076455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bchc5" event={"ID":"a1efedd5-d848-44af-bacc-5e2ce0e56a4e","Type":"ContainerStarted","Data":"e4576d9a1583dbc8fc27543369b013774722a6c2f9ef12830192cb755c37709c"} Jul 21 20:41:28.169854 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.169830 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e3a63c7-1fa2-460d-a186-1f649e81d662-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.169961 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.169858 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e3a63c7-1fa2-460d-a186-1f649e81d662-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.169961 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.169891 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.169961 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.169947 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e3a63c7-1fa2-460d-a186-1f649e81d662-config-out\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170002 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-web-config\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170045 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170107 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170402 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2jsb\" (UniqueName: \"kubernetes.io/projected/1e3a63c7-1fa2-460d-a186-1f649e81d662-kube-api-access-z2jsb\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170402 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170191 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e3a63c7-1fa2-460d-a186-1f649e81d662-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.170402 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.170257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e3a63c7-1fa2-460d-a186-1f649e81d662-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271501 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e3a63c7-1fa2-460d-a186-1f649e81d662-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271501 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271482 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e3a63c7-1fa2-460d-a186-1f649e81d662-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271623 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271503 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e3a63c7-1fa2-460d-a186-1f649e81d662-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271623 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e3a63c7-1fa2-460d-a186-1f649e81d662-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271722 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271722 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271651 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e3a63c7-1fa2-460d-a186-1f649e81d662-config-out\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271722 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.271880 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-web-config\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.271838 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272045 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1e3a63c7-1fa2-460d-a186-1f649e81d662-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272268 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272308 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2jsb\" (UniqueName: \"kubernetes.io/projected/1e3a63c7-1fa2-460d-a186-1f649e81d662-kube-api-access-z2jsb\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272761 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e3a63c7-1fa2-460d-a186-1f649e81d662-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.273039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.272870 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e3a63c7-1fa2-460d-a186-1f649e81d662-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.275199 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.275172 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.275417 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.275397 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e3a63c7-1fa2-460d-a186-1f649e81d662-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.275899 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.276511 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.276635 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-web-config\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.276817 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e3a63c7-1fa2-460d-a186-1f649e81d662-config-out\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.276954 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.277880 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.279812 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.279599 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1e3a63c7-1fa2-460d-a186-1f649e81d662-config-volume\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.282282 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.282264 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2jsb\" (UniqueName: \"kubernetes.io/projected/1e3a63c7-1fa2-460d-a186-1f649e81d662-kube-api-access-z2jsb\") pod \"alertmanager-main-0\" (UID: \"1e3a63c7-1fa2-460d-a186-1f649e81d662\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.296882 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.296864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 20:41:28.449317 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.449293 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 20:41:28.451681 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:28.451651 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3a63c7_1fa2_460d_a186_1f649e81d662.slice/crio-dccdc9e37bba502cba7d7733e631252584937e9185e32a10a73c36c847cc8fd8 WatchSource:0}: Error finding container dccdc9e37bba502cba7d7733e631252584937e9185e32a10a73c36c847cc8fd8: Status 404 returned error can't find the container with id dccdc9e37bba502cba7d7733e631252584937e9185e32a10a73c36c847cc8fd8 Jul 21 20:41:28.675416 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.675338 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:28.677940 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.677907 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7cefcae-d372-4dd8-9ced-d12b33cbce11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-7sqrp\" (UID: \"b7cefcae-d372-4dd8-9ced-d12b33cbce11\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:28.723075 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.723033 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" Jul 21 20:41:28.888384 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:28.888361 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp"] Jul 21 20:41:28.890899 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:28.890794 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cefcae_d372_4dd8_9ced_d12b33cbce11.slice/crio-44e0ea6c3e24bc27ef70c267536b05fbc62ad2d23d04d6c614ef05c904473ed7 WatchSource:0}: Error finding container 44e0ea6c3e24bc27ef70c267536b05fbc62ad2d23d04d6c614ef05c904473ed7: Status 404 returned error can't find the container with id 44e0ea6c3e24bc27ef70c267536b05fbc62ad2d23d04d6c614ef05c904473ed7 Jul 21 20:41:29.080974 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:29.080947 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"dccdc9e37bba502cba7d7733e631252584937e9185e32a10a73c36c847cc8fd8"} Jul 21 20:41:29.082801 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:29.082779 2576 generic.go:356] "Generic (PLEG): container finished" podID="a1efedd5-d848-44af-bacc-5e2ce0e56a4e" containerID="b452382ba693376f01c99431c15b5c1b576945e8028bbb2f2aa8a2bdddb04752" exitCode=0 Jul 21 20:41:29.082898 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:29.082867 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bchc5" event={"ID":"a1efedd5-d848-44af-bacc-5e2ce0e56a4e","Type":"ContainerDied","Data":"b452382ba693376f01c99431c15b5c1b576945e8028bbb2f2aa8a2bdddb04752"} Jul 21 20:41:29.084971 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:29.084949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" event={"ID":"b7cefcae-d372-4dd8-9ced-d12b33cbce11","Type":"ContainerStarted","Data":"11e1e0f3b0d279824aa0812a9d83866aeae98b1e9470005ad9dffc31639f1a22"} Jul 21 20:41:29.085074 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:29.084995 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" event={"ID":"b7cefcae-d372-4dd8-9ced-d12b33cbce11","Type":"ContainerStarted","Data":"7ad8289f0353fdc6e5a8267c00e7448ca7e97bbb333645d4a354866b5ca6c15f"} Jul 21 20:41:29.085074 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:29.085012 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" event={"ID":"b7cefcae-d372-4dd8-9ced-d12b33cbce11","Type":"ContainerStarted","Data":"44e0ea6c3e24bc27ef70c267536b05fbc62ad2d23d04d6c614ef05c904473ed7"} Jul 21 20:41:30.090994 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.090962 2576 generic.go:356] "Generic (PLEG): container finished" podID="1e3a63c7-1fa2-460d-a186-1f649e81d662" containerID="e5a28dcf6c48af4d1ac74371a489ca7227e534e905f7cdc55aeb01727e122126" exitCode=0 Jul 21 20:41:30.091374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.091056 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerDied","Data":"e5a28dcf6c48af4d1ac74371a489ca7227e534e905f7cdc55aeb01727e122126"} Jul 21 20:41:30.093692 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.093668 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bchc5" event={"ID":"a1efedd5-d848-44af-bacc-5e2ce0e56a4e","Type":"ContainerStarted","Data":"fcfa6eaa3991301cc672593c8eef01fbcf9adb45be6c9aa5127b79f0e9aa3970"} Jul 21 20:41:30.093820 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.093717 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bchc5" event={"ID":"a1efedd5-d848-44af-bacc-5e2ce0e56a4e","Type":"ContainerStarted","Data":"c7a9018ae24cdd7167c534b663e54c576f46461f6b38d4f5cdc2e3e942bad2f5"} Jul 21 20:41:30.215557 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.214514 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bchc5" podStartSLOduration=3.184543532 podStartE2EDuration="4.214492821s" totalImagesPullingTime="957.856351ms" totalInitContainerRuntime="72.092938ms" isStatefulPod=true podCreationTimestamp="2026-07-21 20:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:30.209403131 +0000 UTC m=+89.063473860" watchObservedRunningTime="2026-07-21 20:41:30.214492821 +0000 UTC m=+89.068563549" Jul 21 20:41:30.217750 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.217723 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b4d6b796b-97rj4"] Jul 21 20:41:30.222934 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.222910 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.227952 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.227731 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 21 20:41:30.227952 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.227759 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 21 20:41:30.228594 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.228198 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bnfgm\"" Jul 21 20:41:30.228594 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.228304 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 21 20:41:30.228594 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.228419 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 21 20:41:30.229298 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.229278 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 21 20:41:30.229590 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.229354 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9743qdj22pe1i\"" Jul 21 20:41:30.258658 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.258633 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b4d6b796b-97rj4"] Jul 21 20:41:30.391004 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.390978 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kpxc\" (UniqueName: \"kubernetes.io/projected/4b9d673e-952b-4fc3-9d4d-f152748281ca-kube-api-access-2kpxc\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391093 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391025 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391093 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391058 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-grpc-tls\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391093 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391241 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b9d673e-952b-4fc3-9d4d-f152748281ca-metrics-client-ca\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391241 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391192 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-tls\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391314 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.391348 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.391329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492461 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kpxc\" (UniqueName: \"kubernetes.io/projected/4b9d673e-952b-4fc3-9d4d-f152748281ca-kube-api-access-2kpxc\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492586 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492474 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492586 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-grpc-tls\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492586 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492586 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492537 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b9d673e-952b-4fc3-9d4d-f152748281ca-metrics-client-ca\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492586 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-tls\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492839 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.492839 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.492776 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.493411 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.493382 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b9d673e-952b-4fc3-9d4d-f152748281ca-metrics-client-ca\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.495560 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.495310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.495560 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.495310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.495560 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.495513 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.495787 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.495766 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-grpc-tls\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.495850 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.495787 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-tls\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.495958 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.495934 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b9d673e-952b-4fc3-9d4d-f152748281ca-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.502465 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.502414 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kpxc\" (UniqueName: \"kubernetes.io/projected/4b9d673e-952b-4fc3-9d4d-f152748281ca-kube-api-access-2kpxc\") pod \"thanos-querier-5b4d6b796b-97rj4\" (UID: \"4b9d673e-952b-4fc3-9d4d-f152748281ca\") " pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.539307 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.539286 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:30.678892 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:30.678862 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b4d6b796b-97rj4"] Jul 21 20:41:30.683877 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:41:30.683848 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9d673e_952b_4fc3_9d4d_f152748281ca.slice/crio-90969a839faadf3e6762dabf85456a1e7cdc6f39817d1453b82dc0af765ffb19 WatchSource:0}: Error finding container 90969a839faadf3e6762dabf85456a1e7cdc6f39817d1453b82dc0af765ffb19: Status 404 returned error can't find the container with id 90969a839faadf3e6762dabf85456a1e7cdc6f39817d1453b82dc0af765ffb19 Jul 21 20:41:31.100346 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:31.100309 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" event={"ID":"b7cefcae-d372-4dd8-9ced-d12b33cbce11","Type":"ContainerStarted","Data":"784bbb5799b4d5f8598e789ae66ab39c3a611d3a889d9aa034ebd2cb6bed7b42"} Jul 21 20:41:31.101836 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:31.101807 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"90969a839faadf3e6762dabf85456a1e7cdc6f39817d1453b82dc0af765ffb19"} Jul 21 20:41:31.131841 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:31.131576 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-7sqrp" podStartSLOduration=3.884665709 podStartE2EDuration="5.131558674s" totalImagesPullingTime="1.246892965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:31.130563756 +0000 UTC m=+89.984634520" watchObservedRunningTime="2026-07-21 20:41:31.131558674 +0000 UTC m=+89.985629405" Jul 21 20:41:33.113479 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:33.113423 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"f8c48a34d5bcdf705985c547786be9775d585acccc67522aa44cd3da4788fc35"} Jul 21 20:41:33.114018 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:33.113555 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"2d37848413a2d0590c42e75c56af7cc6993041a0d192b5375d1cbf8a08aa0cab"} Jul 21 20:41:33.114018 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:33.113584 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"3fe6e2aec3cbc538cdcb6697b43e81a42c48ce59a92d346c499a1c5c166d1e35"} Jul 21 20:41:33.114018 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:33.113613 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"b93b02f8626ca31120374c60ae146a06f3cb4b9b8224774fb879154ac1f84a7c"} Jul 21 20:41:33.114018 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:33.113638 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"3e3c2fee4150caf407b88cf237f79e1ab3705a70df913d11b35358a086fbd0b5"} Jul 21 20:41:34.120858 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:34.120830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1e3a63c7-1fa2-460d-a186-1f649e81d662","Type":"ContainerStarted","Data":"be3c49c9781f32cc0aadd898182d7dc9713df65549c553b309854fa2b0631e40"} Jul 21 20:41:34.123092 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:34.123068 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"48ed88da3887dffe633e7b8595388f87dcc470bb4c5be1920261dbc3a885e50c"} Jul 21 20:41:34.123208 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:34.123136 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"7087b7aa1d780a57228998a097bbc262d79269311c921ea00793363ed921a7f8"} Jul 21 20:41:34.123208 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:34.123176 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"71e18e0d0303cfa2352281f45d46bd972f897b43f9519c2dab67fee58247e651"} Jul 21 20:41:34.156477 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:34.156412 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.617643565 podStartE2EDuration="7.156397304s" totalImagesPullingTime="4.524809074s" totalInitContainerRuntime="13.944665ms" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:34.154701165 +0000 UTC m=+93.008771915" watchObservedRunningTime="2026-07-21 20:41:34.156397304 +0000 UTC m=+93.010468031" Jul 21 20:41:35.129666 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:35.129628 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"84b53ed6b5f0f2b3a06918e94cd0955f49ac36633f4398945066e48d54e79cae"} Jul 21 20:41:35.130005 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:35.129697 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"75ab80858b93ec7466d89d5514004d44204a956e3d0b2be82912e2ec3dea66b5"} Jul 21 20:41:35.130005 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:35.129722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" event={"ID":"4b9d673e-952b-4fc3-9d4d-f152748281ca","Type":"ContainerStarted","Data":"ef3f6ea1ea1ca08331279c4bf747870f793e24421e613d219918d3bd6c5d3a80"} Jul 21 20:41:35.164392 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:35.164342 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" podStartSLOduration=2.021690275 podStartE2EDuration="5.164325564s" totalImagesPullingTime="3.142635289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:35.163286397 +0000 UTC m=+94.017357141" watchObservedRunningTime="2026-07-21 20:41:35.164325564 +0000 UTC m=+94.018396291" Jul 21 20:41:36.133608 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:36.133578 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:42.143170 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:42.143143 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b4d6b796b-97rj4" Jul 21 20:41:43.018889 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:41:43.018858 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b62zm" Jul 21 20:42:51.715679 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.715651 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-bbcbb5f86-s6l96"] Jul 21 20:42:51.719974 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.719958 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.724311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.724277 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5ntlp\"" Jul 21 20:42:51.724311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.724286 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 21 20:42:51.724513 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.724324 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 21 20:42:51.724513 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.724417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 21 20:42:51.724513 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.724429 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 21 20:42:51.726747 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.726728 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 21 20:42:51.730623 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.730604 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 21 20:42:51.753023 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.753001 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bbcbb5f86-s6l96"] Jul 21 20:42:51.828355 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828312 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-secret-telemeter-client\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828472 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-federate-client-tls\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828472 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828563 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828622 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-serving-certs-ca-bundle\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828720 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828694 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-metrics-client-ca\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828779 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828762 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84bzg\" (UniqueName: \"kubernetes.io/projected/a4865f26-dc8e-4787-b54f-e99309141c9d-kube-api-access-84bzg\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.828832 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.828819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-telemeter-client-tls\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929589 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929569 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-secret-telemeter-client\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929686 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-federate-client-tls\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929686 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929686 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929810 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929702 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-serving-certs-ca-bundle\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929810 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929732 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-metrics-client-ca\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929810 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929756 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bzg\" (UniqueName: \"kubernetes.io/projected/a4865f26-dc8e-4787-b54f-e99309141c9d-kube-api-access-84bzg\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.929810 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.929794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-telemeter-client-tls\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.930539 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.930514 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-metrics-client-ca\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.930629 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.930533 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-serving-certs-ca-bundle\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.930629 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.930578 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4865f26-dc8e-4787-b54f-e99309141c9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.932187 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.932151 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-secret-telemeter-client\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.932321 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.932303 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-federate-client-tls\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.932530 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.932511 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.932615 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.932597 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a4865f26-dc8e-4787-b54f-e99309141c9d-telemeter-client-tls\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:51.938746 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:51.938723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84bzg\" (UniqueName: \"kubernetes.io/projected/a4865f26-dc8e-4787-b54f-e99309141c9d-kube-api-access-84bzg\") pod \"telemeter-client-bbcbb5f86-s6l96\" (UID: \"a4865f26-dc8e-4787-b54f-e99309141c9d\") " pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:52.029048 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:52.029005 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" Jul 21 20:42:52.162731 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:52.162648 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bbcbb5f86-s6l96"] Jul 21 20:42:52.165540 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:42:52.165515 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4865f26_dc8e_4787_b54f_e99309141c9d.slice/crio-213e3a93cf58f3377a7dbd4795ae087d2e25796ff1117a424fe7f4b1fbb3b55c WatchSource:0}: Error finding container 213e3a93cf58f3377a7dbd4795ae087d2e25796ff1117a424fe7f4b1fbb3b55c: Status 404 returned error can't find the container with id 213e3a93cf58f3377a7dbd4795ae087d2e25796ff1117a424fe7f4b1fbb3b55c Jul 21 20:42:52.409857 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:52.409797 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" event={"ID":"a4865f26-dc8e-4787-b54f-e99309141c9d","Type":"ContainerStarted","Data":"213e3a93cf58f3377a7dbd4795ae087d2e25796ff1117a424fe7f4b1fbb3b55c"} Jul 21 20:42:54.422121 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:54.422089 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" event={"ID":"a4865f26-dc8e-4787-b54f-e99309141c9d","Type":"ContainerStarted","Data":"ab5e2ba9563a9c2c11ec59eee8082f4fc57a55e0f884cbcf5998b0340971369a"} Jul 21 20:42:55.427796 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:55.427754 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" event={"ID":"a4865f26-dc8e-4787-b54f-e99309141c9d","Type":"ContainerStarted","Data":"3571b83ee79c147791bf4d04aa6b1ba9d762d8c0df8136654a5669b50c5904b5"} Jul 21 20:42:55.428173 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:55.427820 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" event={"ID":"a4865f26-dc8e-4787-b54f-e99309141c9d","Type":"ContainerStarted","Data":"fc0b39092f0bdbd2041078020f49adcfe391ec2fd5ae120524825b1307021e3f"} Jul 21 20:42:55.469698 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:42:55.469654 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-bbcbb5f86-s6l96" podStartSLOduration=2.284143524 podStartE2EDuration="4.469639607s" totalImagesPullingTime="2.185496083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:42:55.467230594 +0000 UTC m=+174.321301347" watchObservedRunningTime="2026-07-21 20:42:55.469639607 +0000 UTC m=+174.323710333" Jul 21 20:44:19.372116 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.372089 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bp44t"] Jul 21 20:44:19.375158 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.375141 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.378075 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.378057 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wvrbq\"" Jul 21 20:44:19.378130 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.378076 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 20:44:19.378648 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.378634 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 20:44:19.392078 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.392055 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bp44t"] Jul 21 20:44:19.427537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.427504 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf0cde7-7ddb-407f-98d6-a341e07c0c1f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bp44t\" (UID: \"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.427630 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.427553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzr8j\" (UniqueName: \"kubernetes.io/projected/dbf0cde7-7ddb-407f-98d6-a341e07c0c1f-kube-api-access-dzr8j\") pod \"cert-manager-webhook-6486ffd997-bp44t\" (UID: \"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.528525 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.528495 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf0cde7-7ddb-407f-98d6-a341e07c0c1f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bp44t\" (UID: \"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.528634 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.528547 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzr8j\" (UniqueName: \"kubernetes.io/projected/dbf0cde7-7ddb-407f-98d6-a341e07c0c1f-kube-api-access-dzr8j\") pod \"cert-manager-webhook-6486ffd997-bp44t\" (UID: \"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.544556 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.544536 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzr8j\" (UniqueName: \"kubernetes.io/projected/dbf0cde7-7ddb-407f-98d6-a341e07c0c1f-kube-api-access-dzr8j\") pod \"cert-manager-webhook-6486ffd997-bp44t\" (UID: \"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.544665 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.544572 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbf0cde7-7ddb-407f-98d6-a341e07c0c1f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bp44t\" (UID: \"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.684581 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.684514 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:19.810837 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:19.810813 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bp44t"] Jul 21 20:44:19.812829 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:44:19.812794 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbf0cde7_7ddb_407f_98d6_a341e07c0c1f.slice/crio-f05d1d456d5450fbf4330206c7e48c6e2622a5f166b50f3cfd2c796ba9708d79 WatchSource:0}: Error finding container f05d1d456d5450fbf4330206c7e48c6e2622a5f166b50f3cfd2c796ba9708d79: Status 404 returned error can't find the container with id f05d1d456d5450fbf4330206c7e48c6e2622a5f166b50f3cfd2c796ba9708d79 Jul 21 20:44:20.724299 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:20.724209 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" event={"ID":"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f","Type":"ContainerStarted","Data":"f05d1d456d5450fbf4330206c7e48c6e2622a5f166b50f3cfd2c796ba9708d79"} Jul 21 20:44:23.740132 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:23.740058 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" event={"ID":"dbf0cde7-7ddb-407f-98d6-a341e07c0c1f","Type":"ContainerStarted","Data":"32bd01e9873bff04c55378238118592a38443f41fe0a75519905c7e219c7ea52"} Jul 21 20:44:23.740132 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:23.740115 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:23.758890 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:23.758847 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" podStartSLOduration=1.5071919010000001 podStartE2EDuration="4.758835186s" totalImagesPullingTime="3.251643285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:44:23.758091302 +0000 UTC m=+262.612162055" watchObservedRunningTime="2026-07-21 20:44:23.758835186 +0000 UTC m=+262.612905913" Jul 21 20:44:29.745993 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:29.745965 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bp44t" Jul 21 20:44:54.039470 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.039420 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr"] Jul 21 20:44:54.043021 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.042993 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.047379 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.047355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 20:44:54.047761 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.047745 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 20:44:54.047761 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.047751 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rvl9p\"" Jul 21 20:44:54.048195 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.048178 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 20:44:54.053385 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.053367 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 20:44:54.070651 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.070630 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr"] Jul 21 20:44:54.173956 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.173932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94b99f44-1287-4604-9a75-063a178fa4e4-tmp\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.174094 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.173968 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqplj\" (UniqueName: \"kubernetes.io/projected/94b99f44-1287-4604-9a75-063a178fa4e4-kube-api-access-sqplj\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.174094 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.173994 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/94b99f44-1287-4604-9a75-063a178fa4e4-manifests\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.174203 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.174100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94b99f44-1287-4604-9a75-063a178fa4e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.174203 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.174149 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/94b99f44-1287-4604-9a75-063a178fa4e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.275289 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.275269 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94b99f44-1287-4604-9a75-063a178fa4e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.275399 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.275297 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/94b99f44-1287-4604-9a75-063a178fa4e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.275399 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.275342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94b99f44-1287-4604-9a75-063a178fa4e4-tmp\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.275500 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.275477 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqplj\" (UniqueName: \"kubernetes.io/projected/94b99f44-1287-4604-9a75-063a178fa4e4-kube-api-access-sqplj\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.275537 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.275505 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/94b99f44-1287-4604-9a75-063a178fa4e4-manifests\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.275925 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.275904 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/94b99f44-1287-4604-9a75-063a178fa4e4-manifests\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.277550 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.277524 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94b99f44-1287-4604-9a75-063a178fa4e4-tmp\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.277699 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.277683 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94b99f44-1287-4604-9a75-063a178fa4e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.277796 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.277779 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/94b99f44-1287-4604-9a75-063a178fa4e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.284546 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.284525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqplj\" (UniqueName: \"kubernetes.io/projected/94b99f44-1287-4604-9a75-063a178fa4e4-kube-api-access-sqplj\") pod \"opendatahub-operator-controller-manager-647bdf6675-68hjr\" (UID: \"94b99f44-1287-4604-9a75-063a178fa4e4\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.356011 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.355966 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:54.491202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.491171 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr"] Jul 21 20:44:54.494894 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:44:54.494858 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b99f44_1287_4604_9a75_063a178fa4e4.slice/crio-7355e8496a1fff04d505722e50ec4ec9f24fd253f4108c2dd79f98918e6e79c5 WatchSource:0}: Error finding container 7355e8496a1fff04d505722e50ec4ec9f24fd253f4108c2dd79f98918e6e79c5: Status 404 returned error can't find the container with id 7355e8496a1fff04d505722e50ec4ec9f24fd253f4108c2dd79f98918e6e79c5 Jul 21 20:44:54.847320 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:54.847292 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" event={"ID":"94b99f44-1287-4604-9a75-063a178fa4e4","Type":"ContainerStarted","Data":"7355e8496a1fff04d505722e50ec4ec9f24fd253f4108c2dd79f98918e6e79c5"} Jul 21 20:44:57.861972 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:57.861939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" event={"ID":"94b99f44-1287-4604-9a75-063a178fa4e4","Type":"ContainerStarted","Data":"fd04c1a0dd304be0186c00ae2412586b1897f3cc60b8e2040737355cc91f89ad"} Jul 21 20:44:58.867293 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:58.867257 2576 generic.go:356] "Generic (PLEG): container finished" podID="94b99f44-1287-4604-9a75-063a178fa4e4" containerID="fd04c1a0dd304be0186c00ae2412586b1897f3cc60b8e2040737355cc91f89ad" exitCode=0 Jul 21 20:44:58.867600 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:58.867346 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" event={"ID":"94b99f44-1287-4604-9a75-063a178fa4e4","Type":"ContainerDied","Data":"fd04c1a0dd304be0186c00ae2412586b1897f3cc60b8e2040737355cc91f89ad"} Jul 21 20:44:59.873561 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:59.873522 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" event={"ID":"94b99f44-1287-4604-9a75-063a178fa4e4","Type":"ContainerStarted","Data":"7f892949200596ae8a5a4de61aa17c5f2ccd412bd155ee35943b1379a76537b3"} Jul 21 20:44:59.873939 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:59.873666 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:44:59.931158 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:44:59.931103 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" podStartSLOduration=1.443238552 podStartE2EDuration="5.931084802s" totalImagesPullingTime="3.470891565s" totalInitContainerRuntime="1.016954685s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:44:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:44:59.930297343 +0000 UTC m=+298.784368108" watchObservedRunningTime="2026-07-21 20:44:59.931084802 +0000 UTC m=+298.785155530" Jul 21 20:45:01.500677 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:01.500648 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 20:45:10.880069 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:10.880041 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-68hjr" Jul 21 20:45:14.934684 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:14.934651 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw"] Jul 21 20:45:14.937395 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:14.937374 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:14.940059 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:14.940022 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 21 20:45:14.940204 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:14.940176 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-tls\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" Jul 21 20:45:14.940338 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:14.940230 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-65495b7cc9-6pzgw\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" podUID="dd6189de-f4ca-49b2-8c07-3cbd3942f6f5" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:14.940512 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:14.940492 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 21 20:45:14.940512 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:14.940502 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 21 20:45:14.978721 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:14.978668 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw"] Jul 21 20:45:15.032610 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.032579 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.032758 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.032621 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tmp\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.032758 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.032708 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzk27\" (UniqueName: \"kubernetes.io/projected/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-kube-api-access-xzk27\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.051117 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.051088 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7xpb6"] Jul 21 20:45:15.053893 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.053871 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.063454 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.063415 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 21 20:45:15.063572 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.063479 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4p8sd\"" Jul 21 20:45:15.133313 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.133280 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.133496 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.133317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tmp\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.133496 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.133364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzk27\" (UniqueName: \"kubernetes.io/projected/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-kube-api-access-xzk27\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.135533 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.135513 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tmp\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:15.185219 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.185142 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7xpb6"] Jul 21 20:45:15.234234 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.234205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-cert\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.234387 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.234317 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv5j8\" (UniqueName: \"kubernetes.io/projected/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-kube-api-access-tv5j8\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.335697 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.335665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv5j8\" (UniqueName: \"kubernetes.io/projected/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-kube-api-access-tv5j8\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.335866 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.335743 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-cert\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.335866 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:15.335857 2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 21 20:45:15.335969 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:15.335920 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-cert podName:b6de02f7-c3a7-48c1-9f6b-cc99fd54f096 nodeName:}" failed. No retries permitted until 2026-07-21 20:45:15.835898989 +0000 UTC m=+314.689969697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-cert") pod "odh-model-controller-7998c67dc4-7xpb6" (UID: "b6de02f7-c3a7-48c1-9f6b-cc99fd54f096") : secret "odh-model-controller-webhook-cert" not found Jul 21 20:45:15.358598 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.358571 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv5j8\" (UniqueName: \"kubernetes.io/projected/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-kube-api-access-tv5j8\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.832276 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.832244 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 20:45:15.840928 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.840901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-cert\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.843323 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.843289 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6de02f7-c3a7-48c1-9f6b-cc99fd54f096-cert\") pod \"odh-model-controller-7998c67dc4-7xpb6\" (UID: \"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096\") " pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:15.845017 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.844997 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 20:45:15.966562 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:15.966534 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:16.034021 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.033993 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 20:45:16.045426 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.045396 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzk27\" (UniqueName: \"kubernetes.io/projected/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-kube-api-access-xzk27\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:16.115062 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.114988 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7xpb6"] Jul 21 20:45:16.118288 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:45:16.118262 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6de02f7_c3a7_48c1_9f6b_cc99fd54f096.slice/crio-0d0cf7a6e5acd4505f8496d27f30779deaeb66ec7bbf01cd8d75613f1df3fef5 WatchSource:0}: Error finding container 0d0cf7a6e5acd4505f8496d27f30779deaeb66ec7bbf01cd8d75613f1df3fef5: Status 404 returned error can't find the container with id 0d0cf7a6e5acd4505f8496d27f30779deaeb66ec7bbf01cd8d75613f1df3fef5 Jul 21 20:45:16.119623 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.119605 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 20:45:16.133808 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:16.133789 2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: failed to sync secret cache: timed out waiting for the condition Jul 21 20:45:16.133895 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:16.133863 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tls-certs podName:dd6189de-f4ca-49b2-8c07-3cbd3942f6f5 nodeName:}" failed. No retries permitted until 2026-07-21 20:45:16.633848381 +0000 UTC m=+315.487919091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tls-certs") pod "kube-auth-proxy-65495b7cc9-6pzgw" (UID: "dd6189de-f4ca-49b2-8c07-3cbd3942f6f5") : failed to sync secret cache: timed out waiting for the condition Jul 21 20:45:16.307314 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.307283 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 20:45:16.647853 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.647820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:16.650216 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.650192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd6189de-f4ca-49b2-8c07-3cbd3942f6f5-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-6pzgw\" (UID: \"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:16.748718 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.748685 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" Jul 21 20:45:16.840272 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.840235 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v2dtx"] Jul 21 20:45:16.843161 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.843126 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:16.846510 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.846490 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 21 20:45:16.896836 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.895176 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v2dtx"] Jul 21 20:45:16.939804 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.939681 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" event={"ID":"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096","Type":"ContainerStarted","Data":"0d0cf7a6e5acd4505f8496d27f30779deaeb66ec7bbf01cd8d75613f1df3fef5"} Jul 21 20:45:16.951804 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.951609 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e205eb3-a779-478c-8069-82f621d3cb7b-tls-certs\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:16.951804 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.951719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4l9t\" (UniqueName: \"kubernetes.io/projected/9e205eb3-a779-478c-8069-82f621d3cb7b-kube-api-access-k4l9t\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:16.991964 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:16.991921 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw"] Jul 21 20:45:17.052970 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.052943 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4l9t\" (UniqueName: \"kubernetes.io/projected/9e205eb3-a779-478c-8069-82f621d3cb7b-kube-api-access-k4l9t\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:17.053079 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.053007 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e205eb3-a779-478c-8069-82f621d3cb7b-tls-certs\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:17.053124 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:17.053102 2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 21 20:45:17.053176 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:17.053166 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e205eb3-a779-478c-8069-82f621d3cb7b-tls-certs podName:9e205eb3-a779-478c-8069-82f621d3cb7b nodeName:}" failed. No retries permitted until 2026-07-21 20:45:17.553149861 +0000 UTC m=+316.407220566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9e205eb3-a779-478c-8069-82f621d3cb7b-tls-certs") pod "model-serving-api-54d89bd5fc-v2dtx" (UID: "9e205eb3-a779-478c-8069-82f621d3cb7b") : secret "model-serving-api-tls" not found Jul 21 20:45:17.076194 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.076165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4l9t\" (UniqueName: \"kubernetes.io/projected/9e205eb3-a779-478c-8069-82f621d3cb7b-kube-api-access-k4l9t\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:17.556981 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.556927 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e205eb3-a779-478c-8069-82f621d3cb7b-tls-certs\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:17.561150 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.561069 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e205eb3-a779-478c-8069-82f621d3cb7b-tls-certs\") pod \"model-serving-api-54d89bd5fc-v2dtx\" (UID: \"9e205eb3-a779-478c-8069-82f621d3cb7b\") " pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:17.760459 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.760406 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:17.952374 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.952272 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" event={"ID":"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5","Type":"ContainerStarted","Data":"f35575dc2256c8d8f81e8c0bc8e995ecb0fb62f45d897337e9e0857c17596950"} Jul 21 20:45:17.996719 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:17.996686 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-k29ng"] Jul 21 20:45:18.002155 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.002109 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.005008 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.004844 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 20:45:18.005008 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.004863 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 20:45:18.005008 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.004919 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 20:45:18.005008 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.004875 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 20:45:18.005545 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.005523 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jd7dm\"" Jul 21 20:45:18.022864 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.022837 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-k29ng"] Jul 21 20:45:18.162687 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162646 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-local-certs\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.162867 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.162867 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162753 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-cacerts\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.162867 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162784 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.162867 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162826 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-token\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.162867 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162860 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frh2h\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-kube-api-access-frh2h\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.163118 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.162940 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264134 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-local-certs\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264197 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264230 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-cacerts\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264290 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-token\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frh2h\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-kube-api-access-frh2h\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.265202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.264946 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.267717 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.267687 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.268305 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.268257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-local-certs\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.268518 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.268477 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-cacerts\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.269947 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.269924 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.278753 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.278709 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-token\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.279366 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.279342 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frh2h\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-kube-api-access-frh2h\") pod \"istiod-openshift-gateway-7574459656-k29ng\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:18.316774 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:18.316751 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:20.023797 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.023741 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v2dtx"] Jul 21 20:45:20.026719 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:45:20.026672 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e205eb3_a779_478c_8069_82f621d3cb7b.slice/crio-a9de0d70ff05ebb91479ce368e429052404070f23bbd1a18bec1335423a7cdaa WatchSource:0}: Error finding container a9de0d70ff05ebb91479ce368e429052404070f23bbd1a18bec1335423a7cdaa: Status 404 returned error can't find the container with id a9de0d70ff05ebb91479ce368e429052404070f23bbd1a18bec1335423a7cdaa Jul 21 20:45:20.090286 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.090248 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-k29ng"] Jul 21 20:45:20.093663 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:45:20.093636 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56de9a2_8d2f_4c5d_804e_70f3a6ce028e.slice/crio-82ec491a466937695f88a9960885c7d53469ef02d4eb3de683697c1127b1f45e WatchSource:0}: Error finding container 82ec491a466937695f88a9960885c7d53469ef02d4eb3de683697c1127b1f45e: Status 404 returned error can't find the container with id 82ec491a466937695f88a9960885c7d53469ef02d4eb3de683697c1127b1f45e Jul 21 20:45:20.970738 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.970692 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" event={"ID":"9e205eb3-a779-478c-8069-82f621d3cb7b","Type":"ContainerStarted","Data":"a9de0d70ff05ebb91479ce368e429052404070f23bbd1a18bec1335423a7cdaa"} Jul 21 20:45:20.972187 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.972152 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" event={"ID":"b6de02f7-c3a7-48c1-9f6b-cc99fd54f096","Type":"ContainerStarted","Data":"80f1037d148b51db6e3ee65f3cce17492ada47299687ad8e6266d85453c8fbb2"} Jul 21 20:45:20.972349 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.972251 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:20.973678 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.973639 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" event={"ID":"dd6189de-f4ca-49b2-8c07-3cbd3942f6f5","Type":"ContainerStarted","Data":"ebdaace3a166bed248cf6a8c616e780f43d1eb05e235c131149442f111fb63db"} Jul 21 20:45:20.974868 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.974847 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" event={"ID":"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e","Type":"ContainerStarted","Data":"82ec491a466937695f88a9960885c7d53469ef02d4eb3de683697c1127b1f45e"} Jul 21 20:45:20.990545 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:20.990506 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" podStartSLOduration=3.185535271 podStartE2EDuration="6.990492799s" totalImagesPullingTime="3.804957528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:20.988693835 +0000 UTC m=+319.842764558" watchObservedRunningTime="2026-07-21 20:45:20.990492799 +0000 UTC m=+319.844563525" Jul 21 20:45:21.005900 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:21.005845 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-6pzgw" podStartSLOduration=4.09481979 podStartE2EDuration="7.005828236s" totalImagesPullingTime="2.911008446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:21.005593315 +0000 UTC m=+319.859664056" watchObservedRunningTime="2026-07-21 20:45:21.005828236 +0000 UTC m=+319.859898965" Jul 21 20:45:23.451923 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.451889 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 20:45:23.452165 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.451958 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 21 20:45:23.742733 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.742700 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb"] Jul 21 20:45:23.746031 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.746012 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:23.748851 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.748832 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 20:45:23.749596 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.749579 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 20:45:23.749857 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.749821 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 20:45:23.749966 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.749826 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 20:45:23.749966 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.749886 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sj469\"" Jul 21 20:45:23.749966 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.749866 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 20:45:23.756833 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.756811 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb"] Jul 21 20:45:23.914829 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.914793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-cert\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:23.915024 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.914847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-metrics-cert\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:23.915024 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.914922 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdwb4\" (UniqueName: \"kubernetes.io/projected/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-kube-api-access-bdwb4\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:23.915121 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.915074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-manager-config\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:23.992161 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.992121 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" event={"ID":"9e205eb3-a779-478c-8069-82f621d3cb7b","Type":"ContainerStarted","Data":"0dffe7871dc2ea5f7e27a16c0bd9aa20f51fac1c8ff0c42e275c6762af81b891"} Jul 21 20:45:23.992342 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.992221 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:23.993983 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.993921 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" event={"ID":"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e","Type":"ContainerStarted","Data":"0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2"} Jul 21 20:45:23.994079 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:23.994056 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:24.016086 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.016056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdwb4\" (UniqueName: \"kubernetes.io/projected/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-kube-api-access-bdwb4\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.016202 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.016189 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-manager-config\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.016302 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.016277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-cert\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.016426 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.016409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-metrics-cert\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.017103 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.017078 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-manager-config\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.019307 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.019275 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-metrics-cert\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.019427 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.019394 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" podStartSLOduration=4.602947328 podStartE2EDuration="8.019379493s" totalImagesPullingTime="3.416432165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:24.017266799 +0000 UTC m=+322.871337548" watchObservedRunningTime="2026-07-21 20:45:24.019379493 +0000 UTC m=+322.873450226" Jul 21 20:45:24.019506 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.019491 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-cert\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.026280 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.026249 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdwb4\" (UniqueName: \"kubernetes.io/projected/1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7-kube-api-access-bdwb4\") pod \"lws-controller-manager-66f9b7665f-9tjvb\" (UID: \"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7\") " pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.056159 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.056132 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:24.061032 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.060986 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" podStartSLOduration=3.705234765 podStartE2EDuration="7.060971158s" totalImagesPullingTime="3.355736393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:24.059412903 +0000 UTC m=+322.913483652" watchObservedRunningTime="2026-07-21 20:45:24.060971158 +0000 UTC m=+322.915041886" Jul 21 20:45:24.277791 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:24.277763 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb"] Jul 21 20:45:24.280482 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:45:24.280430 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6d6f46_07a5_4929_9dd7_e4ec3262b0b7.slice/crio-75795480f296c90c47e556f3dd6d76288fdb47dcab9fc8d85f7c65acfbd8aa82 WatchSource:0}: Error finding container 75795480f296c90c47e556f3dd6d76288fdb47dcab9fc8d85f7c65acfbd8aa82: Status 404 returned error can't find the container with id 75795480f296c90c47e556f3dd6d76288fdb47dcab9fc8d85f7c65acfbd8aa82 Jul 21 20:45:25.002057 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:25.001980 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" event={"ID":"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7","Type":"ContainerStarted","Data":"75795480f296c90c47e556f3dd6d76288fdb47dcab9fc8d85f7c65acfbd8aa82"} Jul 21 20:45:25.004243 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:25.004147 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-k29ng container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 20:45:25.004243 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:25.004211 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" podUID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 20:45:27.012800 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:27.012772 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" event={"ID":"1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7","Type":"ContainerStarted","Data":"31f9dbde443833c8fbe87a8bd0731b72261898bb1efef4ad8d016325a519e24d"} Jul 21 20:45:27.013153 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:27.012839 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:45:27.944551 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:27.944497 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" podStartSLOduration=2.586674581 podStartE2EDuration="4.944479611s" totalImagesPullingTime="2.35780503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:27.065631886 +0000 UTC m=+325.919702612" watchObservedRunningTime="2026-07-21 20:45:27.944479611 +0000 UTC m=+326.798550337" Jul 21 20:45:27.944952 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:27.944915 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-k29ng"] Jul 21 20:45:27.945133 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:27.945105 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" podUID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" containerName="discovery" containerID="cri-o://0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2" gracePeriod=30 Jul 21 20:45:27.946045 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:27.945904 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:28.195193 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.195141 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:28.254164 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254141 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-token\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254300 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254172 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-dns-cert\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254300 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254195 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-kubeconfig\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254300 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254221 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-cacerts\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254300 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254244 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-ca-configmap\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254300 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254272 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frh2h\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-kube-api-access-frh2h\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254585 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254337 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-local-certs\") pod \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\" (UID: \"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e\") " Jul 21 20:45:28.254766 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.254716 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 20:45:28.256679 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.256648 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-kube-api-access-frh2h" (OuterVolumeSpecName: "kube-api-access-frh2h") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "kube-api-access-frh2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:45:28.256790 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.256728 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:45:28.256846 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.256820 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 20:45:28.256900 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.256838 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-cacerts" (OuterVolumeSpecName: "cacerts") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 20:45:28.256900 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.256871 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 20:45:28.256977 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.256962 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-local-certs" (OuterVolumeSpecName: "local-certs") pod "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" (UID: "d56de9a2-8d2f-4c5d-804e-70f3a6ce028e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 20:45:28.355743 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355720 2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-local-certs\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:28.355743 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355742 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-token\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:28.355872 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355752 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-dns-cert\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:28.355872 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355761 2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-kubeconfig\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:28.355872 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355770 2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-cacerts\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:28.355872 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355779 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-istio-csr-ca-configmap\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:28.355872 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:28.355788 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frh2h\" (UniqueName: \"kubernetes.io/projected/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e-kube-api-access-frh2h\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:45:29.024733 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.024695 2576 generic.go:356] "Generic (PLEG): container finished" podID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" containerID="0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2" exitCode=0 Jul 21 20:45:29.024901 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.024740 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" Jul 21 20:45:29.024901 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.024779 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" event={"ID":"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e","Type":"ContainerDied","Data":"0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2"} Jul 21 20:45:29.024901 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.024827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-k29ng" event={"ID":"d56de9a2-8d2f-4c5d-804e-70f3a6ce028e","Type":"ContainerDied","Data":"82ec491a466937695f88a9960885c7d53469ef02d4eb3de683697c1127b1f45e"} Jul 21 20:45:29.024901 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.024861 2576 scope.go:117] "RemoveContainer" containerID="0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2" Jul 21 20:45:29.033932 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.033913 2576 scope.go:117] "RemoveContainer" containerID="0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2" Jul 21 20:45:29.034196 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:45:29.034177 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2\": container with ID starting with 0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2 not found: ID does not exist" containerID="0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2" Jul 21 20:45:29.034272 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.034206 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2"} err="failed to get container status \"0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2\": rpc error: code = NotFound desc = could not find container \"0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2\": container with ID starting with 0235121346ef1cb5d8c05b9ea818baeb47b2253401daf76d6cdd52c3f14cb3d2 not found: ID does not exist" Jul 21 20:45:29.068823 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.068790 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-k29ng"] Jul 21 20:45:29.074589 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.074562 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-k29ng"] Jul 21 20:45:29.708452 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:29.708411 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" path="/var/lib/kubelet/pods/d56de9a2-8d2f-4c5d-804e-70f3a6ce028e/volumes" Jul 21 20:45:31.982396 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:31.982367 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7xpb6" Jul 21 20:45:35.007709 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:35.007680 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-v2dtx" Jul 21 20:45:38.021159 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:45:38.021126 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66f9b7665f-9tjvb" Jul 21 20:47:04.004546 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.004511 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5"] Jul 21 20:47:04.004959 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.004839 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" containerName="discovery" Jul 21 20:47:04.004959 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.004851 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" containerName="discovery" Jul 21 20:47:04.004959 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.004916 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d56de9a2-8d2f-4c5d-804e-70f3a6ce028e" containerName="discovery" Jul 21 20:47:04.006778 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.006763 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.009723 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.009704 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 20:47:04.010226 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.010210 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 21 20:47:04.010346 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.010330 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 20:47:04.010556 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.010539 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tlcgm\"" Jul 21 20:47:04.013587 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.013572 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 21 20:47:04.024679 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.024664 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5"] Jul 21 20:47:04.052572 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.052552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b461c54-a82c-4396-a0c9-f9b60b104634-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.052677 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.052619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k968\" (UniqueName: \"kubernetes.io/projected/6b461c54-a82c-4396-a0c9-f9b60b104634-kube-api-access-8k968\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.052677 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.052663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b461c54-a82c-4396-a0c9-f9b60b104634-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.153643 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.153620 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b461c54-a82c-4396-a0c9-f9b60b104634-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.153739 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.153674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k968\" (UniqueName: \"kubernetes.io/projected/6b461c54-a82c-4396-a0c9-f9b60b104634-kube-api-access-8k968\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.153739 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.153702 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b461c54-a82c-4396-a0c9-f9b60b104634-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.154189 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.154168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b461c54-a82c-4396-a0c9-f9b60b104634-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.155924 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.155905 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b461c54-a82c-4396-a0c9-f9b60b104634-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.164769 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.164746 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k968\" (UniqueName: \"kubernetes.io/projected/6b461c54-a82c-4396-a0c9-f9b60b104634-kube-api-access-8k968\") pod \"kuadrant-console-plugin-66b5774985-9q6x5\" (UID: \"6b461c54-a82c-4396-a0c9-f9b60b104634\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.316050 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.315991 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" Jul 21 20:47:04.442397 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:04.442367 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5"] Jul 21 20:47:04.445646 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:47:04.445618 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b461c54_a82c_4396_a0c9_f9b60b104634.slice/crio-44b692df39487e7dae4a71d158bf2d75bdb918d6fcecff545c0654c4110dd358 WatchSource:0}: Error finding container 44b692df39487e7dae4a71d158bf2d75bdb918d6fcecff545c0654c4110dd358: Status 404 returned error can't find the container with id 44b692df39487e7dae4a71d158bf2d75bdb918d6fcecff545c0654c4110dd358 Jul 21 20:47:05.434554 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:05.434522 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" event={"ID":"6b461c54-a82c-4396-a0c9-f9b60b104634","Type":"ContainerStarted","Data":"44b692df39487e7dae4a71d158bf2d75bdb918d6fcecff545c0654c4110dd358"} Jul 21 20:47:26.534998 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:26.534932 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" event={"ID":"6b461c54-a82c-4396-a0c9-f9b60b104634","Type":"ContainerStarted","Data":"2684b6d2e851a729da0c41a77e6ddb17f61c0d59ecbb96062f7407dab6234a21"} Jul 21 20:47:26.552960 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:26.552912 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9q6x5" podStartSLOduration=1.771561349 podStartE2EDuration="23.552898716s" totalImagesPullingTime="21.781337367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:26.551982547 +0000 UTC m=+445.406053285" watchObservedRunningTime="2026-07-21 20:47:26.552898716 +0000 UTC m=+445.406969469" Jul 21 20:47:49.264551 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.264509 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-chd25"] Jul 21 20:47:49.764691 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.764664 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-chd25"] Jul 21 20:47:49.764881 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.764789 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:49.767186 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.767164 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bx9zp\"" Jul 21 20:47:49.770646 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.770628 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7cqbt"] Jul 21 20:47:49.788789 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.788770 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7cqbt"] Jul 21 20:47:49.788885 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.788831 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:47:49.813488 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.813463 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs7jz\" (UniqueName: \"kubernetes.io/projected/15441ac8-09dc-4cc9-a01b-8eceb7005777-kube-api-access-gs7jz\") pod \"authorino-75d556bd77-7cqbt\" (UID: \"15441ac8-09dc-4cc9-a01b-8eceb7005777\") " pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:47:49.813594 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.813577 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjbh5\" (UniqueName: \"kubernetes.io/projected/f5a92664-5866-441a-bec7-c8f6bfd1982e-kube-api-access-wjbh5\") pod \"authorino-58cd955675-chd25\" (UID: \"f5a92664-5866-441a-bec7-c8f6bfd1982e\") " pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:49.914579 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.914553 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjbh5\" (UniqueName: \"kubernetes.io/projected/f5a92664-5866-441a-bec7-c8f6bfd1982e-kube-api-access-wjbh5\") pod \"authorino-58cd955675-chd25\" (UID: \"f5a92664-5866-441a-bec7-c8f6bfd1982e\") " pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:49.914668 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.914594 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jz\" (UniqueName: \"kubernetes.io/projected/15441ac8-09dc-4cc9-a01b-8eceb7005777-kube-api-access-gs7jz\") pod \"authorino-75d556bd77-7cqbt\" (UID: \"15441ac8-09dc-4cc9-a01b-8eceb7005777\") " pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:47:49.923482 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.923464 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs7jz\" (UniqueName: \"kubernetes.io/projected/15441ac8-09dc-4cc9-a01b-8eceb7005777-kube-api-access-gs7jz\") pod \"authorino-75d556bd77-7cqbt\" (UID: \"15441ac8-09dc-4cc9-a01b-8eceb7005777\") " pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:47:49.923601 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:49.923582 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjbh5\" (UniqueName: \"kubernetes.io/projected/f5a92664-5866-441a-bec7-c8f6bfd1982e-kube-api-access-wjbh5\") pod \"authorino-58cd955675-chd25\" (UID: \"f5a92664-5866-441a-bec7-c8f6bfd1982e\") " pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:50.073817 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:50.073765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:50.096515 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:50.096494 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:47:50.231656 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:50.231617 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-chd25"] Jul 21 20:47:50.235088 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:47:50.235060 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5a92664_5866_441a_bec7_c8f6bfd1982e.slice/crio-53f0029c4929c24982d85f0ce3504f6afacee652a192d4d7d337432afeeee0d1 WatchSource:0}: Error finding container 53f0029c4929c24982d85f0ce3504f6afacee652a192d4d7d337432afeeee0d1: Status 404 returned error can't find the container with id 53f0029c4929c24982d85f0ce3504f6afacee652a192d4d7d337432afeeee0d1 Jul 21 20:47:50.248786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:50.248765 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7cqbt"] Jul 21 20:47:50.250769 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:47:50.250745 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15441ac8_09dc_4cc9_a01b_8eceb7005777.slice/crio-99a125740cb7f664d5be9152e3e17ac3df2557ba260328a4476a2b6f9a9de0cf WatchSource:0}: Error finding container 99a125740cb7f664d5be9152e3e17ac3df2557ba260328a4476a2b6f9a9de0cf: Status 404 returned error can't find the container with id 99a125740cb7f664d5be9152e3e17ac3df2557ba260328a4476a2b6f9a9de0cf Jul 21 20:47:50.643983 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:50.643947 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7cqbt" event={"ID":"15441ac8-09dc-4cc9-a01b-8eceb7005777","Type":"ContainerStarted","Data":"99a125740cb7f664d5be9152e3e17ac3df2557ba260328a4476a2b6f9a9de0cf"} Jul 21 20:47:50.645115 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:50.645093 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-chd25" event={"ID":"f5a92664-5866-441a-bec7-c8f6bfd1982e","Type":"ContainerStarted","Data":"53f0029c4929c24982d85f0ce3504f6afacee652a192d4d7d337432afeeee0d1"} Jul 21 20:47:54.669466 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:54.669416 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7cqbt" event={"ID":"15441ac8-09dc-4cc9-a01b-8eceb7005777","Type":"ContainerStarted","Data":"dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28"} Jul 21 20:47:54.670784 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:54.670764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-chd25" event={"ID":"f5a92664-5866-441a-bec7-c8f6bfd1982e","Type":"ContainerStarted","Data":"cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4"} Jul 21 20:47:54.683449 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:54.683391 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7cqbt" podStartSLOduration=2.049971913 podStartE2EDuration="5.683379225s" totalImagesPullingTime="3.633407312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:54.682682633 +0000 UTC m=+473.536753359" watchObservedRunningTime="2026-07-21 20:47:54.683379225 +0000 UTC m=+473.537449951" Jul 21 20:47:54.697536 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:54.697489 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-chd25" podStartSLOduration=2.058694513 podStartE2EDuration="5.69747332s" totalImagesPullingTime="3.638778807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:54.69591841 +0000 UTC m=+473.549989164" watchObservedRunningTime="2026-07-21 20:47:54.69747332 +0000 UTC m=+473.551544047" Jul 21 20:47:54.723201 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:54.723178 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-chd25"] Jul 21 20:47:56.679087 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:56.679051 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-chd25" podUID="f5a92664-5866-441a-bec7-c8f6bfd1982e" containerName="authorino" containerID="cri-o://cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4" gracePeriod=30 Jul 21 20:47:57.529584 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.529557 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:57.579898 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.579878 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjbh5\" (UniqueName: \"kubernetes.io/projected/f5a92664-5866-441a-bec7-c8f6bfd1982e-kube-api-access-wjbh5\") pod \"f5a92664-5866-441a-bec7-c8f6bfd1982e\" (UID: \"f5a92664-5866-441a-bec7-c8f6bfd1982e\") " Jul 21 20:47:57.581970 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.581938 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5a92664-5866-441a-bec7-c8f6bfd1982e-kube-api-access-wjbh5" (OuterVolumeSpecName: "kube-api-access-wjbh5") pod "f5a92664-5866-441a-bec7-c8f6bfd1982e" (UID: "f5a92664-5866-441a-bec7-c8f6bfd1982e"). InnerVolumeSpecName "kube-api-access-wjbh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:47:57.680454 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.680377 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjbh5\" (UniqueName: \"kubernetes.io/projected/f5a92664-5866-441a-bec7-c8f6bfd1982e-kube-api-access-wjbh5\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:47:57.684687 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.684665 2576 generic.go:356] "Generic (PLEG): container finished" podID="f5a92664-5866-441a-bec7-c8f6bfd1982e" containerID="cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4" exitCode=0 Jul 21 20:47:57.684789 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.684697 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-chd25" event={"ID":"f5a92664-5866-441a-bec7-c8f6bfd1982e","Type":"ContainerDied","Data":"cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4"} Jul 21 20:47:57.684789 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.684702 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-chd25" Jul 21 20:47:57.684789 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.684720 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-chd25" event={"ID":"f5a92664-5866-441a-bec7-c8f6bfd1982e","Type":"ContainerDied","Data":"53f0029c4929c24982d85f0ce3504f6afacee652a192d4d7d337432afeeee0d1"} Jul 21 20:47:57.684789 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.684736 2576 scope.go:117] "RemoveContainer" containerID="cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4" Jul 21 20:47:57.692929 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.692915 2576 scope.go:117] "RemoveContainer" containerID="cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4" Jul 21 20:47:57.693170 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:47:57.693149 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4\": container with ID starting with cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4 not found: ID does not exist" containerID="cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4" Jul 21 20:47:57.693234 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.693182 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4"} err="failed to get container status \"cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4\": rpc error: code = NotFound desc = could not find container \"cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4\": container with ID starting with cb25758db67b3dc3b63db877421c8fd26c554b780803d68920e00beb0203cdd4 not found: ID does not exist" Jul 21 20:47:57.707780 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.707761 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-chd25"] Jul 21 20:47:57.708576 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:57.708556 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-chd25"] Jul 21 20:47:59.708310 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:47:59.708280 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5a92664-5866-441a-bec7-c8f6bfd1982e" path="/var/lib/kubelet/pods/f5a92664-5866-441a-bec7-c8f6bfd1982e/volumes" Jul 21 20:48:23.185561 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.185533 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p5qx"] Jul 21 20:48:23.185914 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.185869 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5a92664-5866-441a-bec7-c8f6bfd1982e" containerName="authorino" Jul 21 20:48:23.185914 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.185880 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5a92664-5866-441a-bec7-c8f6bfd1982e" containerName="authorino" Jul 21 20:48:23.185987 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.185958 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5a92664-5866-441a-bec7-c8f6bfd1982e" containerName="authorino" Jul 21 20:48:23.192883 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.192864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:23.196455 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.196406 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p5qx"] Jul 21 20:48:23.268468 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.268425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7mww\" (UniqueName: \"kubernetes.io/projected/5a7aa8fb-d257-4936-98f8-eb962fbcc18c-kube-api-access-b7mww\") pod \"authorino-5d4b67c446-7p5qx\" (UID: \"5a7aa8fb-d257-4936-98f8-eb962fbcc18c\") " pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:23.369402 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.369379 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mww\" (UniqueName: \"kubernetes.io/projected/5a7aa8fb-d257-4936-98f8-eb962fbcc18c-kube-api-access-b7mww\") pod \"authorino-5d4b67c446-7p5qx\" (UID: \"5a7aa8fb-d257-4936-98f8-eb962fbcc18c\") " pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:23.377943 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.377918 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7mww\" (UniqueName: \"kubernetes.io/projected/5a7aa8fb-d257-4936-98f8-eb962fbcc18c-kube-api-access-b7mww\") pod \"authorino-5d4b67c446-7p5qx\" (UID: \"5a7aa8fb-d257-4936-98f8-eb962fbcc18c\") " pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:23.436404 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.436355 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p5qx"] Jul 21 20:48:23.436542 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.436530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:23.474841 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.474814 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fc86f844b-jfd45"] Jul 21 20:48:23.478730 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.478708 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:23.490334 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.490287 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fc86f844b-jfd45"] Jul 21 20:48:23.565201 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.565179 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p5qx"] Jul 21 20:48:23.567359 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:48:23.567329 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a7aa8fb_d257_4936_98f8_eb962fbcc18c.slice/crio-6a32e4320601ae283b595ecaa5d95ecc72b59e474d257b0e64fdb9c142c6a22d WatchSource:0}: Error finding container 6a32e4320601ae283b595ecaa5d95ecc72b59e474d257b0e64fdb9c142c6a22d: Status 404 returned error can't find the container with id 6a32e4320601ae283b595ecaa5d95ecc72b59e474d257b0e64fdb9c142c6a22d Jul 21 20:48:23.571510 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.571487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgwdj\" (UniqueName: \"kubernetes.io/projected/7c52cb94-ee2c-4398-8dfc-605030fef878-kube-api-access-fgwdj\") pod \"authorino-5fc86f844b-jfd45\" (UID: \"7c52cb94-ee2c-4398-8dfc-605030fef878\") " pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:23.663691 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.663665 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fc86f844b-jfd45"] Jul 21 20:48:23.663908 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:48:23.663890 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fgwdj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5fc86f844b-jfd45" podUID="7c52cb94-ee2c-4398-8dfc-605030fef878" Jul 21 20:48:23.672156 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.672130 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgwdj\" (UniqueName: \"kubernetes.io/projected/7c52cb94-ee2c-4398-8dfc-605030fef878-kube-api-access-fgwdj\") pod \"authorino-5fc86f844b-jfd45\" (UID: \"7c52cb94-ee2c-4398-8dfc-605030fef878\") " pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:23.684643 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.684623 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgwdj\" (UniqueName: \"kubernetes.io/projected/7c52cb94-ee2c-4398-8dfc-605030fef878-kube-api-access-fgwdj\") pod \"authorino-5fc86f844b-jfd45\" (UID: \"7c52cb94-ee2c-4398-8dfc-605030fef878\") " pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:23.693232 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.693181 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd94997d-mw72t"] Jul 21 20:48:23.698870 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.698854 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.701551 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.701529 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 20:48:23.710641 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.710623 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd94997d-mw72t"] Jul 21 20:48:23.772753 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.772728 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61d47788-4b3c-4b34-ae5d-ec1229773ec6-tls-cert\") pod \"authorino-6dd94997d-mw72t\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.772856 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.772759 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlvrd\" (UniqueName: \"kubernetes.io/projected/61d47788-4b3c-4b34-ae5d-ec1229773ec6-kube-api-access-vlvrd\") pod \"authorino-6dd94997d-mw72t\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.792957 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.792939 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:23.793028 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.792982 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" event={"ID":"5a7aa8fb-d257-4936-98f8-eb962fbcc18c","Type":"ContainerStarted","Data":"6a32e4320601ae283b595ecaa5d95ecc72b59e474d257b0e64fdb9c142c6a22d"} Jul 21 20:48:23.802304 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.802286 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:23.873885 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.873863 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgwdj\" (UniqueName: \"kubernetes.io/projected/7c52cb94-ee2c-4398-8dfc-605030fef878-kube-api-access-fgwdj\") pod \"7c52cb94-ee2c-4398-8dfc-605030fef878\" (UID: \"7c52cb94-ee2c-4398-8dfc-605030fef878\") " Jul 21 20:48:23.874010 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.873995 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61d47788-4b3c-4b34-ae5d-ec1229773ec6-tls-cert\") pod \"authorino-6dd94997d-mw72t\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.874052 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.874027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlvrd\" (UniqueName: \"kubernetes.io/projected/61d47788-4b3c-4b34-ae5d-ec1229773ec6-kube-api-access-vlvrd\") pod \"authorino-6dd94997d-mw72t\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.875846 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.875821 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c52cb94-ee2c-4398-8dfc-605030fef878-kube-api-access-fgwdj" (OuterVolumeSpecName: "kube-api-access-fgwdj") pod "7c52cb94-ee2c-4398-8dfc-605030fef878" (UID: "7c52cb94-ee2c-4398-8dfc-605030fef878"). InnerVolumeSpecName "kube-api-access-fgwdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:48:23.876272 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.876255 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61d47788-4b3c-4b34-ae5d-ec1229773ec6-tls-cert\") pod \"authorino-6dd94997d-mw72t\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.882276 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.882253 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlvrd\" (UniqueName: \"kubernetes.io/projected/61d47788-4b3c-4b34-ae5d-ec1229773ec6-kube-api-access-vlvrd\") pod \"authorino-6dd94997d-mw72t\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:23.975113 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:23.975095 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgwdj\" (UniqueName: \"kubernetes.io/projected/7c52cb94-ee2c-4398-8dfc-605030fef878-kube-api-access-fgwdj\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:48:24.008781 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.008758 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:48:24.170960 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.170927 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd94997d-mw72t"] Jul 21 20:48:24.172687 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:48:24.172664 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61d47788_4b3c_4b34_ae5d_ec1229773ec6.slice/crio-a1dbdca8d5d82c67cd4631df16122ef76af93699384772c13a2a5243b8153c1f WatchSource:0}: Error finding container a1dbdca8d5d82c67cd4631df16122ef76af93699384772c13a2a5243b8153c1f: Status 404 returned error can't find the container with id a1dbdca8d5d82c67cd4631df16122ef76af93699384772c13a2a5243b8153c1f Jul 21 20:48:24.799584 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.799554 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd94997d-mw72t" event={"ID":"61d47788-4b3c-4b34-ae5d-ec1229773ec6","Type":"ContainerStarted","Data":"994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d"} Jul 21 20:48:24.799964 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.799609 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd94997d-mw72t" event={"ID":"61d47788-4b3c-4b34-ae5d-ec1229773ec6","Type":"ContainerStarted","Data":"a1dbdca8d5d82c67cd4631df16122ef76af93699384772c13a2a5243b8153c1f"} Jul 21 20:48:24.801171 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.801146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" event={"ID":"5a7aa8fb-d257-4936-98f8-eb962fbcc18c","Type":"ContainerStarted","Data":"63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84"} Jul 21 20:48:24.801171 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.801165 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fc86f844b-jfd45" Jul 21 20:48:24.801320 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.801227 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" podUID="5a7aa8fb-d257-4936-98f8-eb962fbcc18c" containerName="authorino" containerID="cri-o://63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84" gracePeriod=30 Jul 21 20:48:24.817280 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.817227 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dd94997d-mw72t" podStartSLOduration=1.296116019 podStartE2EDuration="1.817214735s" totalImagesPullingTime="521.098716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:48:24.815304794 +0000 UTC m=+503.669375539" watchObservedRunningTime="2026-07-21 20:48:24.817214735 +0000 UTC m=+503.671285463" Jul 21 20:48:24.834543 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.834496 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" podStartSLOduration=1.419488735 podStartE2EDuration="1.834479718s" totalImagesPullingTime="414.990983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:48:24.833658104 +0000 UTC m=+503.687728831" watchObservedRunningTime="2026-07-21 20:48:24.834479718 +0000 UTC m=+503.688550446" Jul 21 20:48:24.847399 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.847354 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7cqbt"] Jul 21 20:48:24.847690 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.847571 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7cqbt" podUID="15441ac8-09dc-4cc9-a01b-8eceb7005777" containerName="authorino" containerID="cri-o://dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28" gracePeriod=30 Jul 21 20:48:24.861564 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.861542 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fc86f844b-jfd45"] Jul 21 20:48:24.864361 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:24.864329 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5fc86f844b-jfd45"] Jul 21 20:48:25.103946 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.103925 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:25.116113 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.116094 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:48:25.186229 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.186201 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs7jz\" (UniqueName: \"kubernetes.io/projected/15441ac8-09dc-4cc9-a01b-8eceb7005777-kube-api-access-gs7jz\") pod \"15441ac8-09dc-4cc9-a01b-8eceb7005777\" (UID: \"15441ac8-09dc-4cc9-a01b-8eceb7005777\") " Jul 21 20:48:25.186385 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.186287 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7mww\" (UniqueName: \"kubernetes.io/projected/5a7aa8fb-d257-4936-98f8-eb962fbcc18c-kube-api-access-b7mww\") pod \"5a7aa8fb-d257-4936-98f8-eb962fbcc18c\" (UID: \"5a7aa8fb-d257-4936-98f8-eb962fbcc18c\") " Jul 21 20:48:25.188263 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.188232 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15441ac8-09dc-4cc9-a01b-8eceb7005777-kube-api-access-gs7jz" (OuterVolumeSpecName: "kube-api-access-gs7jz") pod "15441ac8-09dc-4cc9-a01b-8eceb7005777" (UID: "15441ac8-09dc-4cc9-a01b-8eceb7005777"). InnerVolumeSpecName "kube-api-access-gs7jz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:48:25.188358 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.188260 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a7aa8fb-d257-4936-98f8-eb962fbcc18c-kube-api-access-b7mww" (OuterVolumeSpecName: "kube-api-access-b7mww") pod "5a7aa8fb-d257-4936-98f8-eb962fbcc18c" (UID: "5a7aa8fb-d257-4936-98f8-eb962fbcc18c"). InnerVolumeSpecName "kube-api-access-b7mww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:48:25.287879 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.287822 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs7jz\" (UniqueName: \"kubernetes.io/projected/15441ac8-09dc-4cc9-a01b-8eceb7005777-kube-api-access-gs7jz\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:48:25.287879 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.287845 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7mww\" (UniqueName: \"kubernetes.io/projected/5a7aa8fb-d257-4936-98f8-eb962fbcc18c-kube-api-access-b7mww\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:48:25.714418 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.714383 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c52cb94-ee2c-4398-8dfc-605030fef878" path="/var/lib/kubelet/pods/7c52cb94-ee2c-4398-8dfc-605030fef878/volumes" Jul 21 20:48:25.806346 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.806319 2576 generic.go:356] "Generic (PLEG): container finished" podID="5a7aa8fb-d257-4936-98f8-eb962fbcc18c" containerID="63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84" exitCode=0 Jul 21 20:48:25.806689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.806369 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" Jul 21 20:48:25.806689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.806379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" event={"ID":"5a7aa8fb-d257-4936-98f8-eb962fbcc18c","Type":"ContainerDied","Data":"63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84"} Jul 21 20:48:25.806689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.806423 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7p5qx" event={"ID":"5a7aa8fb-d257-4936-98f8-eb962fbcc18c","Type":"ContainerDied","Data":"6a32e4320601ae283b595ecaa5d95ecc72b59e474d257b0e64fdb9c142c6a22d"} Jul 21 20:48:25.806689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.806465 2576 scope.go:117] "RemoveContainer" containerID="63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84" Jul 21 20:48:25.807876 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.807661 2576 generic.go:356] "Generic (PLEG): container finished" podID="15441ac8-09dc-4cc9-a01b-8eceb7005777" containerID="dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28" exitCode=0 Jul 21 20:48:25.807876 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.807714 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7cqbt" event={"ID":"15441ac8-09dc-4cc9-a01b-8eceb7005777","Type":"ContainerDied","Data":"dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28"} Jul 21 20:48:25.807876 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.807748 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7cqbt" event={"ID":"15441ac8-09dc-4cc9-a01b-8eceb7005777","Type":"ContainerDied","Data":"99a125740cb7f664d5be9152e3e17ac3df2557ba260328a4476a2b6f9a9de0cf"} Jul 21 20:48:25.807876 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.807721 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7cqbt" Jul 21 20:48:25.814828 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.814813 2576 scope.go:117] "RemoveContainer" containerID="63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84" Jul 21 20:48:25.815067 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:48:25.815044 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84\": container with ID starting with 63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84 not found: ID does not exist" containerID="63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84" Jul 21 20:48:25.815145 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.815069 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84"} err="failed to get container status \"63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84\": rpc error: code = NotFound desc = could not find container \"63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84\": container with ID starting with 63f5efb0d163f98a84561a1cc5c10c60e6642eb55f0dd9daac4c196a12799e84 not found: ID does not exist" Jul 21 20:48:25.815145 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.815085 2576 scope.go:117] "RemoveContainer" containerID="dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28" Jul 21 20:48:25.822068 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.822052 2576 scope.go:117] "RemoveContainer" containerID="dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28" Jul 21 20:48:25.822302 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:48:25.822285 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28\": container with ID starting with dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28 not found: ID does not exist" containerID="dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28" Jul 21 20:48:25.822359 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.822311 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28"} err="failed to get container status \"dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28\": rpc error: code = NotFound desc = could not find container \"dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28\": container with ID starting with dede5eabcf9827727ff4c9f1a7614f357d5da513981201fd068f73208a52da28 not found: ID does not exist" Jul 21 20:48:25.825297 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.825276 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p5qx"] Jul 21 20:48:25.830720 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.830702 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7p5qx"] Jul 21 20:48:25.842143 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.842120 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7cqbt"] Jul 21 20:48:25.847455 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:25.847422 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7cqbt"] Jul 21 20:48:27.708606 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:27.708574 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15441ac8-09dc-4cc9-a01b-8eceb7005777" path="/var/lib/kubelet/pods/15441ac8-09dc-4cc9-a01b-8eceb7005777/volumes" Jul 21 20:48:27.708995 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:48:27.708889 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a7aa8fb-d257-4936-98f8-eb962fbcc18c" path="/var/lib/kubelet/pods/5a7aa8fb-d257-4936-98f8-eb962fbcc18c/volumes" Jul 21 20:49:06.969694 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.969663 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-qmcc6"] Jul 21 20:49:06.970169 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.970139 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7aa8fb-d257-4936-98f8-eb962fbcc18c" containerName="authorino" Jul 21 20:49:06.970169 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.970157 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7aa8fb-d257-4936-98f8-eb962fbcc18c" containerName="authorino" Jul 21 20:49:06.970281 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.970206 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15441ac8-09dc-4cc9-a01b-8eceb7005777" containerName="authorino" Jul 21 20:49:06.970281 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.970216 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15441ac8-09dc-4cc9-a01b-8eceb7005777" containerName="authorino" Jul 21 20:49:06.970372 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.970316 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7aa8fb-d257-4936-98f8-eb962fbcc18c" containerName="authorino" Jul 21 20:49:06.970372 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.970329 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15441ac8-09dc-4cc9-a01b-8eceb7005777" containerName="authorino" Jul 21 20:49:06.974337 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.974317 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:06.977355 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.977333 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 21 20:49:06.977567 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.977547 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-t89sk\"" Jul 21 20:49:06.995250 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:06.995222 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-qmcc6"] Jul 21 20:49:07.092225 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.092196 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54gt6\" (UniqueName: \"kubernetes.io/projected/5e14f08e-dd79-47ed-9676-ffddd218ad93-kube-api-access-54gt6\") pod \"payload-processing-5bf47cd988-qmcc6\" (UID: \"5e14f08e-dd79-47ed-9676-ffddd218ad93\") " pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.092401 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.092292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e14f08e-dd79-47ed-9676-ffddd218ad93-plugins-config-volume\") pod \"payload-processing-5bf47cd988-qmcc6\" (UID: \"5e14f08e-dd79-47ed-9676-ffddd218ad93\") " pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.192776 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.192756 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54gt6\" (UniqueName: \"kubernetes.io/projected/5e14f08e-dd79-47ed-9676-ffddd218ad93-kube-api-access-54gt6\") pod \"payload-processing-5bf47cd988-qmcc6\" (UID: \"5e14f08e-dd79-47ed-9676-ffddd218ad93\") " pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.192897 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.192813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e14f08e-dd79-47ed-9676-ffddd218ad93-plugins-config-volume\") pod \"payload-processing-5bf47cd988-qmcc6\" (UID: \"5e14f08e-dd79-47ed-9676-ffddd218ad93\") " pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.193420 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.193402 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e14f08e-dd79-47ed-9676-ffddd218ad93-plugins-config-volume\") pod \"payload-processing-5bf47cd988-qmcc6\" (UID: \"5e14f08e-dd79-47ed-9676-ffddd218ad93\") " pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.207566 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.207541 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54gt6\" (UniqueName: \"kubernetes.io/projected/5e14f08e-dd79-47ed-9676-ffddd218ad93-kube-api-access-54gt6\") pod \"payload-processing-5bf47cd988-qmcc6\" (UID: \"5e14f08e-dd79-47ed-9676-ffddd218ad93\") " pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.285416 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.285367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:07.440003 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.439976 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-qmcc6"] Jul 21 20:49:07.446756 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:49:07.446732 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e14f08e_dd79_47ed_9676_ffddd218ad93.slice/crio-800375034b7d7b419d38ba63971fe6d282423fd224347ce81a29abdc1ba43011 WatchSource:0}: Error finding container 800375034b7d7b419d38ba63971fe6d282423fd224347ce81a29abdc1ba43011: Status 404 returned error can't find the container with id 800375034b7d7b419d38ba63971fe6d282423fd224347ce81a29abdc1ba43011 Jul 21 20:49:07.983452 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:07.983397 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" event={"ID":"5e14f08e-dd79-47ed-9676-ffddd218ad93","Type":"ContainerStarted","Data":"800375034b7d7b419d38ba63971fe6d282423fd224347ce81a29abdc1ba43011"} Jul 21 20:49:09.993674 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:09.993638 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" event={"ID":"5e14f08e-dd79-47ed-9676-ffddd218ad93","Type":"ContainerStarted","Data":"59dc70deb38c96c7e4bf009786ab0c3c222e0382ae9fdb42f2bfd913cb5a9baf"} Jul 21 20:49:09.994060 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:09.993716 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:10.016039 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:10.015994 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" podStartSLOduration=2.43816211 podStartE2EDuration="4.015980912s" totalImagesPullingTime="1.577818802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:49:10.015194828 +0000 UTC m=+548.869265577" watchObservedRunningTime="2026-07-21 20:49:10.015980912 +0000 UTC m=+548.870051638" Jul 21 20:49:20.999394 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:20.999366 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-qmcc6" Jul 21 20:49:39.994075 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:39.994037 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zs656"] Jul 21 20:49:39.997458 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:39.997424 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.000803 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.000778 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 20:49:40.000910 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.000832 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 21 20:49:40.000910 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.000848 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 20:49:40.000910 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.000857 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k4xb8\"" Jul 21 20:49:40.009332 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.009306 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zs656"] Jul 21 20:49:40.038246 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.038224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxflm\" (UniqueName: \"kubernetes.io/projected/b703768a-6c96-4939-9257-d788324e5dc0-kube-api-access-jxflm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.038337 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.038254 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b703768a-6c96-4939-9257-d788324e5dc0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.038337 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.038274 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.038423 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.038351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.038423 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.038374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.038423 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.038402 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139360 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139495 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxflm\" (UniqueName: \"kubernetes.io/projected/b703768a-6c96-4939-9257-d788324e5dc0-kube-api-access-jxflm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139658 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b703768a-6c96-4939-9257-d788324e5dc0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139658 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139555 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139786 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139765 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139846 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139813 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.139943 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.139925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.141634 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.141611 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b703768a-6c96-4939-9257-d788324e5dc0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.141805 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.141787 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b703768a-6c96-4939-9257-d788324e5dc0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.148759 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.148737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxflm\" (UniqueName: \"kubernetes.io/projected/b703768a-6c96-4939-9257-d788324e5dc0-kube-api-access-jxflm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zs656\" (UID: \"b703768a-6c96-4939-9257-d788324e5dc0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.307513 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.307459 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:40.439703 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:40.439646 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zs656"] Jul 21 20:49:40.441907 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:49:40.441874 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb703768a_6c96_4939_9257_d788324e5dc0.slice/crio-e5957aa8da25db4cbf53b48145da7043fb99897e767fab5ce673e8202beca6af WatchSource:0}: Error finding container e5957aa8da25db4cbf53b48145da7043fb99897e767fab5ce673e8202beca6af: Status 404 returned error can't find the container with id e5957aa8da25db4cbf53b48145da7043fb99897e767fab5ce673e8202beca6af Jul 21 20:49:41.126806 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:41.126768 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" event={"ID":"b703768a-6c96-4939-9257-d788324e5dc0","Type":"ContainerStarted","Data":"e5957aa8da25db4cbf53b48145da7043fb99897e767fab5ce673e8202beca6af"} Jul 21 20:49:42.133829 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:42.133791 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" event={"ID":"b703768a-6c96-4939-9257-d788324e5dc0","Type":"ContainerStarted","Data":"49c8f35534c3782214ab72cb9e595f8c6df776105948f385595a46134f7ddfba"} Jul 21 20:49:42.157362 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:42.157315 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" podStartSLOduration=2.123571989 podStartE2EDuration="3.157300182s" totalImagesPullingTime="1.033728193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:49:42.155058502 +0000 UTC m=+581.009129249" watchObservedRunningTime="2026-07-21 20:49:42.157300182 +0000 UTC m=+581.011370910" Jul 21 20:49:50.308642 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:50.308611 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:50.309007 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:50.308665 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:50.320579 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:50.320554 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:49:51.188408 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:49:51.188384 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zs656" Jul 21 20:50:24.828227 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.828153 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz"] Jul 21 20:50:24.833864 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.833847 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:24.840398 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.840077 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 21 20:50:24.854489 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.854465 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz"] Jul 21 20:50:24.985395 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.985371 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7b7x\" (UniqueName: \"kubernetes.io/projected/eb145dfb-89b8-48b7-85b6-321593d65f9a-kube-api-access-l7b7x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:24.985530 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.985411 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb145dfb-89b8-48b7-85b6-321593d65f9a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:24.985530 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.985474 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:24.985530 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.985499 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:24.985638 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.985553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:24.985638 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:24.985569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086148 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086091 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086148 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086154 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086168 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086215 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7b7x\" (UniqueName: \"kubernetes.io/projected/eb145dfb-89b8-48b7-85b6-321593d65f9a-kube-api-access-l7b7x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb145dfb-89b8-48b7-85b6-321593d65f9a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086624 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086572 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086624 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086590 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.086725 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.086674 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.088269 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.088250 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb145dfb-89b8-48b7-85b6-321593d65f9a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.088557 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.088541 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb145dfb-89b8-48b7-85b6-321593d65f9a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.097913 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.097895 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7b7x\" (UniqueName: \"kubernetes.io/projected/eb145dfb-89b8-48b7-85b6-321593d65f9a-kube-api-access-l7b7x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nr4xz\" (UID: \"eb145dfb-89b8-48b7-85b6-321593d65f9a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.143431 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.143408 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:25.275588 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.275561 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz"] Jul 21 20:50:25.278909 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:50:25.278885 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb145dfb_89b8_48b7_85b6_321593d65f9a.slice/crio-64aa1887fd4a26d3a11d598a04dfb980ecafcf3a55ef1869f28b5033e15daec8 WatchSource:0}: Error finding container 64aa1887fd4a26d3a11d598a04dfb980ecafcf3a55ef1869f28b5033e15daec8: Status 404 returned error can't find the container with id 64aa1887fd4a26d3a11d598a04dfb980ecafcf3a55ef1869f28b5033e15daec8 Jul 21 20:50:25.280215 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.280197 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 20:50:25.321819 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:25.321794 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" event={"ID":"eb145dfb-89b8-48b7-85b6-321593d65f9a","Type":"ContainerStarted","Data":"64aa1887fd4a26d3a11d598a04dfb980ecafcf3a55ef1869f28b5033e15daec8"} Jul 21 20:50:26.327565 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:26.327527 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" event={"ID":"eb145dfb-89b8-48b7-85b6-321593d65f9a","Type":"ContainerStarted","Data":"090476ba30f5a2975694078f2a030f89563343853c8abf1f9ba845f0750c337a"} Jul 21 20:50:26.349268 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:26.349203 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" podStartSLOduration=2.143615441 podStartE2EDuration="2.349184666s" totalImagesPullingTime="205.569225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:50:26.345876157 +0000 UTC m=+625.199946883" watchObservedRunningTime="2026-07-21 20:50:26.349184666 +0000 UTC m=+625.203255394" Jul 21 20:50:35.143778 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:35.143740 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:35.144146 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:35.143826 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:35.155685 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:35.155660 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:35.379974 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:35.379950 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nr4xz" Jul 21 20:50:44.707060 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.707023 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt"] Jul 21 20:50:44.710383 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.710364 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.713599 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.713577 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 20:50:44.721944 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.721920 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt"] Jul 21 20:50:44.734408 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.734386 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.734517 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.734416 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkdbc\" (UniqueName: \"kubernetes.io/projected/707c724d-923e-4a4e-ad6f-f6d846ca3f42-kube-api-access-mkdbc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.734569 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.734516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.734606 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.734592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.734657 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.734621 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/707c724d-923e-4a4e-ad6f-f6d846ca3f42-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.734760 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.734739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835210 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835187 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835221 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835237 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/707c724d-923e-4a4e-ad6f-f6d846ca3f42-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835311 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835298 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835543 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835314 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkdbc\" (UniqueName: \"kubernetes.io/projected/707c724d-923e-4a4e-ad6f-f6d846ca3f42-kube-api-access-mkdbc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835655 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835630 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835717 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835673 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.835754 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.835728 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.837561 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.837543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/707c724d-923e-4a4e-ad6f-f6d846ca3f42-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.837724 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.837707 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/707c724d-923e-4a4e-ad6f-f6d846ca3f42-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:44.846242 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:44.846222 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkdbc\" (UniqueName: \"kubernetes.io/projected/707c724d-923e-4a4e-ad6f-f6d846ca3f42-kube-api-access-mkdbc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rltgt\" (UID: \"707c724d-923e-4a4e-ad6f-f6d846ca3f42\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:45.020582 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:45.020556 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:45.149254 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:45.149225 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt"] Jul 21 20:50:45.152545 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:50:45.152522 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod707c724d_923e_4a4e_ad6f_f6d846ca3f42.slice/crio-1fd0a1357f747d737f71d2e2ce00f5c118d4285c15f85bd5c30e5d142452c7c1 WatchSource:0}: Error finding container 1fd0a1357f747d737f71d2e2ce00f5c118d4285c15f85bd5c30e5d142452c7c1: Status 404 returned error can't find the container with id 1fd0a1357f747d737f71d2e2ce00f5c118d4285c15f85bd5c30e5d142452c7c1 Jul 21 20:50:45.415611 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:45.415583 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" event={"ID":"707c724d-923e-4a4e-ad6f-f6d846ca3f42","Type":"ContainerStarted","Data":"1fd0a1357f747d737f71d2e2ce00f5c118d4285c15f85bd5c30e5d142452c7c1"} Jul 21 20:50:46.421301 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:46.421265 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" event={"ID":"707c724d-923e-4a4e-ad6f-f6d846ca3f42","Type":"ContainerStarted","Data":"5055f720fa30285fc22336e616ce7755d625499eb5ab4c6458576c0719bda367"} Jul 21 20:50:46.439660 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:46.439604 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" podStartSLOduration=2.211392511 podStartE2EDuration="2.439585885s" totalImagesPullingTime="228.193374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:50:46.439011087 +0000 UTC m=+645.293081814" watchObservedRunningTime="2026-07-21 20:50:46.439585885 +0000 UTC m=+645.293656612" Jul 21 20:50:55.021062 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:55.021031 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:55.021515 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:55.021078 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:55.033653 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:55.033626 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:50:55.473093 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:50:55.473065 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rltgt" Jul 21 20:52:11.994193 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:11.994119 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5658b9f6d7-d96qt"] Jul 21 20:52:11.997359 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:11.997342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.010583 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.010560 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5658b9f6d7-d96qt"] Jul 21 20:52:12.099905 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.099881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgp9n\" (UniqueName: \"kubernetes.io/projected/f44f0265-89da-4e7d-8afd-e42618681cd1-kube-api-access-xgp9n\") pod \"authorino-5658b9f6d7-d96qt\" (UID: \"f44f0265-89da-4e7d-8afd-e42618681cd1\") " pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.100021 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.099919 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f44f0265-89da-4e7d-8afd-e42618681cd1-tls-cert\") pod \"authorino-5658b9f6d7-d96qt\" (UID: \"f44f0265-89da-4e7d-8afd-e42618681cd1\") " pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.200443 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.200414 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f44f0265-89da-4e7d-8afd-e42618681cd1-tls-cert\") pod \"authorino-5658b9f6d7-d96qt\" (UID: \"f44f0265-89da-4e7d-8afd-e42618681cd1\") " pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.200560 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.200512 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgp9n\" (UniqueName: \"kubernetes.io/projected/f44f0265-89da-4e7d-8afd-e42618681cd1-kube-api-access-xgp9n\") pod \"authorino-5658b9f6d7-d96qt\" (UID: \"f44f0265-89da-4e7d-8afd-e42618681cd1\") " pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.202743 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.202721 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f44f0265-89da-4e7d-8afd-e42618681cd1-tls-cert\") pod \"authorino-5658b9f6d7-d96qt\" (UID: \"f44f0265-89da-4e7d-8afd-e42618681cd1\") " pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.208981 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.208960 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgp9n\" (UniqueName: \"kubernetes.io/projected/f44f0265-89da-4e7d-8afd-e42618681cd1-kube-api-access-xgp9n\") pod \"authorino-5658b9f6d7-d96qt\" (UID: \"f44f0265-89da-4e7d-8afd-e42618681cd1\") " pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.306130 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.306076 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5658b9f6d7-d96qt" Jul 21 20:52:12.449974 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.449938 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5658b9f6d7-d96qt"] Jul 21 20:52:12.451353 ip-10-0-128-226 kubenswrapper[2576]: W0721 20:52:12.451323 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf44f0265_89da_4e7d_8afd_e42618681cd1.slice/crio-508de1caf4326a4362fefe4b171de77d5da9e30d353c0b1baa3cd2ec8ba54493 WatchSource:0}: Error finding container 508de1caf4326a4362fefe4b171de77d5da9e30d353c0b1baa3cd2ec8ba54493: Status 404 returned error can't find the container with id 508de1caf4326a4362fefe4b171de77d5da9e30d353c0b1baa3cd2ec8ba54493 Jul 21 20:52:12.798380 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:12.798351 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5658b9f6d7-d96qt" event={"ID":"f44f0265-89da-4e7d-8afd-e42618681cd1","Type":"ContainerStarted","Data":"508de1caf4326a4362fefe4b171de77d5da9e30d353c0b1baa3cd2ec8ba54493"} Jul 21 20:52:13.804542 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:13.804496 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5658b9f6d7-d96qt" event={"ID":"f44f0265-89da-4e7d-8afd-e42618681cd1","Type":"ContainerStarted","Data":"36a8bc3b90718128ff874b898a9409102104d3924b056a7f1368cab40d0e034f"} Jul 21 20:52:13.899234 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:13.899180 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5658b9f6d7-d96qt" podStartSLOduration=2.39138057 podStartE2EDuration="2.899163053s" totalImagesPullingTime="507.782483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:52:13.896030549 +0000 UTC m=+732.750101273" watchObservedRunningTime="2026-07-21 20:52:13.899163053 +0000 UTC m=+732.753233803" Jul 21 20:52:14.064621 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.064522 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd94997d-mw72t"] Jul 21 20:52:14.064796 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.064706 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6dd94997d-mw72t" podUID="61d47788-4b3c-4b34-ae5d-ec1229773ec6" containerName="authorino" containerID="cri-o://994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d" gracePeriod=30 Jul 21 20:52:14.342304 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.342274 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:52:14.420888 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.420858 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61d47788-4b3c-4b34-ae5d-ec1229773ec6-tls-cert\") pod \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " Jul 21 20:52:14.421064 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.420938 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlvrd\" (UniqueName: \"kubernetes.io/projected/61d47788-4b3c-4b34-ae5d-ec1229773ec6-kube-api-access-vlvrd\") pod \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\" (UID: \"61d47788-4b3c-4b34-ae5d-ec1229773ec6\") " Jul 21 20:52:14.423193 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.423152 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d47788-4b3c-4b34-ae5d-ec1229773ec6-kube-api-access-vlvrd" (OuterVolumeSpecName: "kube-api-access-vlvrd") pod "61d47788-4b3c-4b34-ae5d-ec1229773ec6" (UID: "61d47788-4b3c-4b34-ae5d-ec1229773ec6"). InnerVolumeSpecName "kube-api-access-vlvrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 20:52:14.431845 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.431813 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61d47788-4b3c-4b34-ae5d-ec1229773ec6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "61d47788-4b3c-4b34-ae5d-ec1229773ec6" (UID: "61d47788-4b3c-4b34-ae5d-ec1229773ec6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 20:52:14.522163 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.522125 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlvrd\" (UniqueName: \"kubernetes.io/projected/61d47788-4b3c-4b34-ae5d-ec1229773ec6-kube-api-access-vlvrd\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:52:14.522163 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.522152 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/61d47788-4b3c-4b34-ae5d-ec1229773ec6-tls-cert\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 20:52:14.810198 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.810167 2576 generic.go:356] "Generic (PLEG): container finished" podID="61d47788-4b3c-4b34-ae5d-ec1229773ec6" containerID="994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d" exitCode=0 Jul 21 20:52:14.810689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.810221 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd94997d-mw72t" Jul 21 20:52:14.810689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.810261 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd94997d-mw72t" event={"ID":"61d47788-4b3c-4b34-ae5d-ec1229773ec6","Type":"ContainerDied","Data":"994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d"} Jul 21 20:52:14.810689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.810302 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd94997d-mw72t" event={"ID":"61d47788-4b3c-4b34-ae5d-ec1229773ec6","Type":"ContainerDied","Data":"a1dbdca8d5d82c67cd4631df16122ef76af93699384772c13a2a5243b8153c1f"} Jul 21 20:52:14.810689 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.810319 2576 scope.go:117] "RemoveContainer" containerID="994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d" Jul 21 20:52:14.819164 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.819150 2576 scope.go:117] "RemoveContainer" containerID="994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d" Jul 21 20:52:14.819412 ip-10-0-128-226 kubenswrapper[2576]: E0721 20:52:14.819394 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d\": container with ID starting with 994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d not found: ID does not exist" containerID="994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d" Jul 21 20:52:14.819511 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.819418 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d"} err="failed to get container status \"994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d\": rpc error: code = NotFound desc = could not find container \"994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d\": container with ID starting with 994a32d8a95c42fd228285048376bd344842e0fa40c03a6dbd80e733f95f190d not found: ID does not exist" Jul 21 20:52:14.841088 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.841062 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd94997d-mw72t"] Jul 21 20:52:14.859094 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:14.859074 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd94997d-mw72t"] Jul 21 20:52:15.708888 ip-10-0-128-226 kubenswrapper[2576]: I0721 20:52:15.708855 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61d47788-4b3c-4b34-ae5d-ec1229773ec6" path="/var/lib/kubelet/pods/61d47788-4b3c-4b34-ae5d-ec1229773ec6/volumes" Jul 21 21:17:30.902543 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.902474 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b"] Jul 21 21:17:30.902930 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.902821 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61d47788-4b3c-4b34-ae5d-ec1229773ec6" containerName="authorino" Jul 21 21:17:30.902930 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.902833 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d47788-4b3c-4b34-ae5d-ec1229773ec6" containerName="authorino" Jul 21 21:17:30.902930 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.902896 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="61d47788-4b3c-4b34-ae5d-ec1229773ec6" containerName="authorino" Jul 21 21:17:30.905834 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.905818 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:30.909102 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.909081 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c15522\"/\"default-dockercfg-bsjzj\"" Jul 21 21:17:30.909218 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.909086 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c15522\"/\"kube-root-ca.crt\"" Jul 21 21:17:30.909218 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.909115 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c15522\"/\"openshift-service-ca.crt\"" Jul 21 21:17:30.909218 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.909164 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c15522\"/\"test-model-c15522-kserve-self-signed-certs\"" Jul 21 21:17:30.917423 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:30.917398 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b"] Jul 21 21:17:31.029192 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.029168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5cst\" (UniqueName: \"kubernetes.io/projected/00db50dd-5d97-479e-a7fd-3f59bdcf3731-kube-api-access-h5cst\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.029304 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.029214 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tls-certs\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.029304 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.029233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-model-cache\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.029304 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.029248 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-dshm\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.029421 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.029320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tmp-dir\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.029421 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.029365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-home\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130459 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130414 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tmp-dir\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130563 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130476 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-home\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130563 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130538 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5cst\" (UniqueName: \"kubernetes.io/projected/00db50dd-5d97-479e-a7fd-3f59bdcf3731-kube-api-access-h5cst\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130681 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130605 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tls-certs\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130681 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-model-cache\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130681 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130660 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-dshm\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130830 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130761 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tmp-dir\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130830 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130789 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-home\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.130936 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.130904 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-model-cache\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.132725 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.132706 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-dshm\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.133153 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.133135 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tls-certs\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.138776 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.138755 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5cst\" (UniqueName: \"kubernetes.io/projected/00db50dd-5d97-479e-a7fd-3f59bdcf3731-kube-api-access-h5cst\") pod \"test-model-c15522-kserve-86f8546797-tbk8b\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.215716 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.215691 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:31.348110 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.348088 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b"] Jul 21 21:17:31.350246 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:17:31.350221 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00db50dd_5d97_479e_a7fd_3f59bdcf3731.slice/crio-b3aa99aae607f63640b2a6b4a5f27ffa7bb8131ca0ae15ea4f5d09ed03553f4b WatchSource:0}: Error finding container b3aa99aae607f63640b2a6b4a5f27ffa7bb8131ca0ae15ea4f5d09ed03553f4b: Status 404 returned error can't find the container with id b3aa99aae607f63640b2a6b4a5f27ffa7bb8131ca0ae15ea4f5d09ed03553f4b Jul 21 21:17:31.572658 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.572576 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" event={"ID":"00db50dd-5d97-479e-a7fd-3f59bdcf3731","Type":"ContainerStarted","Data":"6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342"} Jul 21 21:17:31.572658 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.572615 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" event={"ID":"00db50dd-5d97-479e-a7fd-3f59bdcf3731","Type":"ContainerStarted","Data":"b3aa99aae607f63640b2a6b4a5f27ffa7bb8131ca0ae15ea4f5d09ed03553f4b"} Jul 21 21:17:31.593014 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:31.592968 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" podStartSLOduration=1.59295541 podStartE2EDuration="1.59295541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:17:31.590789162 +0000 UTC m=+2250.444859889" watchObservedRunningTime="2026-07-21 21:17:31.59295541 +0000 UTC m=+2250.447026178" Jul 21 21:17:41.216579 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:41.216551 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:41.217133 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:41.216590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:41.228384 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:41.228354 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:17:41.632563 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:17:41.632488 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:18:10.251493 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:10.251462 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b"] Jul 21 21:18:25.271275 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.271237 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" podUID="00db50dd-5d97-479e-a7fd-3f59bdcf3731" containerName="main" containerID="cri-o://6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342" gracePeriod=45 Jul 21 21:18:25.516240 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.516219 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:18:25.540032 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.539969 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5cst\" (UniqueName: \"kubernetes.io/projected/00db50dd-5d97-479e-a7fd-3f59bdcf3731-kube-api-access-h5cst\") pod \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " Jul 21 21:18:25.540032 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540008 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tls-certs\") pod \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " Jul 21 21:18:25.540205 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540062 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-dshm\") pod \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " Jul 21 21:18:25.540205 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540106 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-home\") pod \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " Jul 21 21:18:25.540205 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540139 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tmp-dir\") pod \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " Jul 21 21:18:25.540205 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540164 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-model-cache\") pod \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\" (UID: \"00db50dd-5d97-479e-a7fd-3f59bdcf3731\") " Jul 21 21:18:25.540412 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540372 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-home" (OuterVolumeSpecName: "home") pod "00db50dd-5d97-479e-a7fd-3f59bdcf3731" (UID: "00db50dd-5d97-479e-a7fd-3f59bdcf3731"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:18:25.540606 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540578 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00db50dd-5d97-479e-a7fd-3f59bdcf3731" (UID: "00db50dd-5d97-479e-a7fd-3f59bdcf3731"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:18:25.540697 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540613 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-model-cache" (OuterVolumeSpecName: "model-cache") pod "00db50dd-5d97-479e-a7fd-3f59bdcf3731" (UID: "00db50dd-5d97-479e-a7fd-3f59bdcf3731"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:18:25.540697 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.540627 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-home\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:18:25.542445 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.542396 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00db50dd-5d97-479e-a7fd-3f59bdcf3731" (UID: "00db50dd-5d97-479e-a7fd-3f59bdcf3731"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:18:25.542595 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.542545 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00db50dd-5d97-479e-a7fd-3f59bdcf3731-kube-api-access-h5cst" (OuterVolumeSpecName: "kube-api-access-h5cst") pod "00db50dd-5d97-479e-a7fd-3f59bdcf3731" (UID: "00db50dd-5d97-479e-a7fd-3f59bdcf3731"). InnerVolumeSpecName "kube-api-access-h5cst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:18:25.542726 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.542704 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-dshm" (OuterVolumeSpecName: "dshm") pod "00db50dd-5d97-479e-a7fd-3f59bdcf3731" (UID: "00db50dd-5d97-479e-a7fd-3f59bdcf3731"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:18:25.641402 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.641380 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5cst\" (UniqueName: \"kubernetes.io/projected/00db50dd-5d97-479e-a7fd-3f59bdcf3731-kube-api-access-h5cst\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:18:25.641402 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.641401 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tls-certs\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:18:25.641546 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.641409 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-dshm\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:18:25.641546 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.641417 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-tmp-dir\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:18:25.641546 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.641425 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00db50dd-5d97-479e-a7fd-3f59bdcf3731-model-cache\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:18:25.813882 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.813823 2576 generic.go:356] "Generic (PLEG): container finished" podID="00db50dd-5d97-479e-a7fd-3f59bdcf3731" containerID="6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342" exitCode=0 Jul 21 21:18:25.813882 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.813858 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" event={"ID":"00db50dd-5d97-479e-a7fd-3f59bdcf3731","Type":"ContainerDied","Data":"6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342"} Jul 21 21:18:25.813882 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.813872 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" Jul 21 21:18:25.813882 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.813882 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b" event={"ID":"00db50dd-5d97-479e-a7fd-3f59bdcf3731","Type":"ContainerDied","Data":"b3aa99aae607f63640b2a6b4a5f27ffa7bb8131ca0ae15ea4f5d09ed03553f4b"} Jul 21 21:18:25.814108 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.813899 2576 scope.go:117] "RemoveContainer" containerID="6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342" Jul 21 21:18:25.821951 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.821933 2576 scope.go:117] "RemoveContainer" containerID="6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342" Jul 21 21:18:25.822218 ip-10-0-128-226 kubenswrapper[2576]: E0721 21:18:25.822196 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342\": container with ID starting with 6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342 not found: ID does not exist" containerID="6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342" Jul 21 21:18:25.822284 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.822229 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342"} err="failed to get container status \"6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342\": rpc error: code = NotFound desc = could not find container \"6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342\": container with ID starting with 6c491cae1a72b04beab144e282f99ce708a9da10bdbdf3998187669c7bfbe342 not found: ID does not exist" Jul 21 21:18:25.831307 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.831286 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b"] Jul 21 21:18:25.835167 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:25.835148 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c15522/test-model-c15522-kserve-86f8546797-tbk8b"] Jul 21 21:18:27.709057 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:18:27.709021 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00db50dd-5d97-479e-a7fd-3f59bdcf3731" path="/var/lib/kubelet/pods/00db50dd-5d97-479e-a7fd-3f59bdcf3731/volumes" Jul 21 21:21:11.018352 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.018323 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj"] Jul 21 21:21:11.020721 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.018754 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00db50dd-5d97-479e-a7fd-3f59bdcf3731" containerName="main" Jul 21 21:21:11.020721 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.018767 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="00db50dd-5d97-479e-a7fd-3f59bdcf3731" containerName="main" Jul 21 21:21:11.020721 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.018828 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="00db50dd-5d97-479e-a7fd-3f59bdcf3731" containerName="main" Jul 21 21:21:11.021601 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.021586 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.023964 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.023940 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"kube-root-ca.crt\"" Jul 21 21:21:11.024766 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.024744 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"openshift-service-ca.crt\"" Jul 21 21:21:11.024852 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.024836 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"auth-test-model-80e9ec-kserve-self-signed-certs\"" Jul 21 21:21:11.024909 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.024897 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"default-dockercfg-j5dgq\"" Jul 21 21:21:11.032273 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.032252 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj"] Jul 21 21:21:11.145987 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.145959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-dshm\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.145987 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.145986 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-tmp-dir\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.146236 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.146011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/008df56c-fb7f-466d-931d-0d6937fdba87-tls-certs\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.146236 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.146112 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-model-cache\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.146236 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.146145 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dnhx\" (UniqueName: \"kubernetes.io/projected/008df56c-fb7f-466d-931d-0d6937fdba87-kube-api-access-4dnhx\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.146236 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.146223 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-home\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247329 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-model-cache\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247476 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247340 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dnhx\" (UniqueName: \"kubernetes.io/projected/008df56c-fb7f-466d-931d-0d6937fdba87-kube-api-access-4dnhx\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247476 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247383 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-home\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247598 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-dshm\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247598 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-tmp-dir\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247598 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/008df56c-fb7f-466d-931d-0d6937fdba87-tls-certs\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247749 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247705 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-model-cache\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247815 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-home\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.247877 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.247827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-tmp-dir\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.249602 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.249574 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-dshm\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.249938 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.249919 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/008df56c-fb7f-466d-931d-0d6937fdba87-tls-certs\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.256037 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.256018 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dnhx\" (UniqueName: \"kubernetes.io/projected/008df56c-fb7f-466d-931d-0d6937fdba87-kube-api-access-4dnhx\") pod \"auth-test-model-80e9ec-kserve-759764f755-b6klj\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.329922 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.329902 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:11.457961 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.457938 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj"] Jul 21 21:21:11.460752 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:21:11.460720 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod008df56c_fb7f_466d_931d_0d6937fdba87.slice/crio-4ee73a3067e5943f09bc3d9f63a030c49c775a0e8d6a8535ba046dda7de81269 WatchSource:0}: Error finding container 4ee73a3067e5943f09bc3d9f63a030c49c775a0e8d6a8535ba046dda7de81269: Status 404 returned error can't find the container with id 4ee73a3067e5943f09bc3d9f63a030c49c775a0e8d6a8535ba046dda7de81269 Jul 21 21:21:11.548074 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.548007 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" event={"ID":"008df56c-fb7f-466d-931d-0d6937fdba87","Type":"ContainerStarted","Data":"74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424"} Jul 21 21:21:11.548074 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:11.548052 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" event={"ID":"008df56c-fb7f-466d-931d-0d6937fdba87","Type":"ContainerStarted","Data":"4ee73a3067e5943f09bc3d9f63a030c49c775a0e8d6a8535ba046dda7de81269"} Jul 21 21:21:21.330709 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:21.330678 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:21.331286 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:21.330726 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:21.342629 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:21.342602 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:21:21.383448 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:21.383400 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" podStartSLOduration=11.383387447 podStartE2EDuration="11.383387447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:21:11.571316263 +0000 UTC m=+2470.425386989" watchObservedRunningTime="2026-07-21 21:21:21.383387447 +0000 UTC m=+2480.237458174" Jul 21 21:21:21.601265 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:21:21.601197 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:23:16.106354 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:16.106315 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj"] Jul 21 21:23:31.125980 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.125937 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" podUID="008df56c-fb7f-466d-931d-0d6937fdba87" containerName="main" containerID="cri-o://74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424" gracePeriod=45 Jul 21 21:23:31.377090 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.377038 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:23:31.454616 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454592 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-model-cache\") pod \"008df56c-fb7f-466d-931d-0d6937fdba87\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " Jul 21 21:23:31.454738 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454622 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-home\") pod \"008df56c-fb7f-466d-931d-0d6937fdba87\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " Jul 21 21:23:31.454738 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454651 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-dshm\") pod \"008df56c-fb7f-466d-931d-0d6937fdba87\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " Jul 21 21:23:31.454738 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454690 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-tmp-dir\") pod \"008df56c-fb7f-466d-931d-0d6937fdba87\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " Jul 21 21:23:31.454841 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454802 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/008df56c-fb7f-466d-931d-0d6937fdba87-tls-certs\") pod \"008df56c-fb7f-466d-931d-0d6937fdba87\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " Jul 21 21:23:31.454841 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454829 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dnhx\" (UniqueName: \"kubernetes.io/projected/008df56c-fb7f-466d-931d-0d6937fdba87-kube-api-access-4dnhx\") pod \"008df56c-fb7f-466d-931d-0d6937fdba87\" (UID: \"008df56c-fb7f-466d-931d-0d6937fdba87\") " Jul 21 21:23:31.454925 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454852 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-home" (OuterVolumeSpecName: "home") pod "008df56c-fb7f-466d-931d-0d6937fdba87" (UID: "008df56c-fb7f-466d-931d-0d6937fdba87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:23:31.454925 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454904 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-model-cache" (OuterVolumeSpecName: "model-cache") pod "008df56c-fb7f-466d-931d-0d6937fdba87" (UID: "008df56c-fb7f-466d-931d-0d6937fdba87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:23:31.455015 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.454994 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "008df56c-fb7f-466d-931d-0d6937fdba87" (UID: "008df56c-fb7f-466d-931d-0d6937fdba87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:23:31.455115 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.455101 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-tmp-dir\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:23:31.455174 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.455117 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-model-cache\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:23:31.455174 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.455126 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-home\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:23:31.456852 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.456829 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-dshm" (OuterVolumeSpecName: "dshm") pod "008df56c-fb7f-466d-931d-0d6937fdba87" (UID: "008df56c-fb7f-466d-931d-0d6937fdba87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:23:31.456965 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.456945 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/008df56c-fb7f-466d-931d-0d6937fdba87-kube-api-access-4dnhx" (OuterVolumeSpecName: "kube-api-access-4dnhx") pod "008df56c-fb7f-466d-931d-0d6937fdba87" (UID: "008df56c-fb7f-466d-931d-0d6937fdba87"). InnerVolumeSpecName "kube-api-access-4dnhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:23:31.457020 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.456987 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/008df56c-fb7f-466d-931d-0d6937fdba87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "008df56c-fb7f-466d-931d-0d6937fdba87" (UID: "008df56c-fb7f-466d-931d-0d6937fdba87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:23:31.555656 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.555629 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/008df56c-fb7f-466d-931d-0d6937fdba87-tls-certs\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:23:31.555656 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.555652 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dnhx\" (UniqueName: \"kubernetes.io/projected/008df56c-fb7f-466d-931d-0d6937fdba87-kube-api-access-4dnhx\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:23:31.555776 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:31.555665 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/008df56c-fb7f-466d-931d-0d6937fdba87-dshm\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:23:32.166860 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.166771 2576 generic.go:356] "Generic (PLEG): container finished" podID="008df56c-fb7f-466d-931d-0d6937fdba87" containerID="74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424" exitCode=0 Jul 21 21:23:32.166860 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.166805 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" Jul 21 21:23:32.167291 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.166863 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" event={"ID":"008df56c-fb7f-466d-931d-0d6937fdba87","Type":"ContainerDied","Data":"74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424"} Jul 21 21:23:32.167291 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.166903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj" event={"ID":"008df56c-fb7f-466d-931d-0d6937fdba87","Type":"ContainerDied","Data":"4ee73a3067e5943f09bc3d9f63a030c49c775a0e8d6a8535ba046dda7de81269"} Jul 21 21:23:32.167291 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.166920 2576 scope.go:117] "RemoveContainer" containerID="74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424" Jul 21 21:23:32.175341 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.175324 2576 scope.go:117] "RemoveContainer" containerID="74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424" Jul 21 21:23:32.175635 ip-10-0-128-226 kubenswrapper[2576]: E0721 21:23:32.175618 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424\": container with ID starting with 74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424 not found: ID does not exist" containerID="74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424" Jul 21 21:23:32.175698 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.175644 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424"} err="failed to get container status \"74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424\": rpc error: code = NotFound desc = could not find container \"74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424\": container with ID starting with 74019d9bcce070a3562724001c8dc407a96b71c45509159a0cf252ecfb2df424 not found: ID does not exist" Jul 21 21:23:32.186373 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.186348 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj"] Jul 21 21:23:32.191122 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:32.191096 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/auth-test-model-80e9ec-kserve-759764f755-b6klj"] Jul 21 21:23:33.708504 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:33.708475 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="008df56c-fb7f-466d-931d-0d6937fdba87" path="/var/lib/kubelet/pods/008df56c-fb7f-466d-931d-0d6937fdba87/volumes" Jul 21 21:23:57.306167 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.306134 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b"] Jul 21 21:23:57.306692 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.306485 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="008df56c-fb7f-466d-931d-0d6937fdba87" containerName="main" Jul 21 21:23:57.306692 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.306496 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="008df56c-fb7f-466d-931d-0d6937fdba87" containerName="main" Jul 21 21:23:57.306692 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.306549 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="008df56c-fb7f-466d-931d-0d6937fdba87" containerName="main" Jul 21 21:23:57.309569 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.309552 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.313009 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.312988 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"kube-root-ca.crt\"" Jul 21 21:23:57.313259 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.313241 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"sub-test-model-80e9ec-kserve-self-signed-certs\"" Jul 21 21:23:57.313376 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.313318 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"openshift-service-ca.crt\"" Jul 21 21:23:57.313376 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.313330 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-80e9ec\"/\"default-dockercfg-j5dgq\"" Jul 21 21:23:57.318579 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.318556 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b"] Jul 21 21:23:57.347868 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.347841 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-model-cache\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.347968 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.347887 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tls-certs\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.347968 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.347920 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-home\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.348092 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.347967 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzxhb\" (UniqueName: \"kubernetes.io/projected/02732677-4194-48e6-9dca-e6cc3cc6bbe6-kube-api-access-rzxhb\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.348092 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.347986 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-dshm\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.348092 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.348006 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tmp-dir\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449112 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-home\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449212 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449128 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzxhb\" (UniqueName: \"kubernetes.io/projected/02732677-4194-48e6-9dca-e6cc3cc6bbe6-kube-api-access-rzxhb\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449212 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449155 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-dshm\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449328 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tmp-dir\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449398 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-model-cache\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449483 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tls-certs\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449558 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449536 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-home\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449626 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449608 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tmp-dir\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.449674 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.449659 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-model-cache\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.451318 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.451300 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-dshm\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.451859 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.451840 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tls-certs\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.457429 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.457402 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzxhb\" (UniqueName: \"kubernetes.io/projected/02732677-4194-48e6-9dca-e6cc3cc6bbe6-kube-api-access-rzxhb\") pod \"sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.619193 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.619135 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:23:57.752343 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:57.752321 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b"] Jul 21 21:23:57.754122 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:23:57.754087 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02732677_4194_48e6_9dca_e6cc3cc6bbe6.slice/crio-246bf93f8adc5ccd8bd44887ac0557ce121ee4d32eaf7262983ca593dc9e5184 WatchSource:0}: Error finding container 246bf93f8adc5ccd8bd44887ac0557ce121ee4d32eaf7262983ca593dc9e5184: Status 404 returned error can't find the container with id 246bf93f8adc5ccd8bd44887ac0557ce121ee4d32eaf7262983ca593dc9e5184 Jul 21 21:23:58.291255 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:58.291224 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" event={"ID":"02732677-4194-48e6-9dca-e6cc3cc6bbe6","Type":"ContainerStarted","Data":"1809a0b2568fa06b9eb43ed1505934b781a28cf52252a5da24d6d4e46e54f30a"} Jul 21 21:23:58.291255 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:58.291261 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" event={"ID":"02732677-4194-48e6-9dca-e6cc3cc6bbe6","Type":"ContainerStarted","Data":"246bf93f8adc5ccd8bd44887ac0557ce121ee4d32eaf7262983ca593dc9e5184"} Jul 21 21:23:58.326760 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:23:58.326716 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" podStartSLOduration=1.326702349 podStartE2EDuration="1.326702349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:23:58.325148968 +0000 UTC m=+2637.179219725" watchObservedRunningTime="2026-07-21 21:23:58.326702349 +0000 UTC m=+2637.180773076" Jul 21 21:24:07.619427 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:07.619397 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:24:07.619811 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:07.619456 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:24:07.631784 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:07.631762 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:24:08.349471 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:08.349428 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:24:36.282059 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:36.282028 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b"] Jul 21 21:24:51.301489 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.301451 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" podUID="02732677-4194-48e6-9dca-e6cc3cc6bbe6" containerName="main" containerID="cri-o://1809a0b2568fa06b9eb43ed1505934b781a28cf52252a5da24d6d4e46e54f30a" gracePeriod=45 Jul 21 21:24:51.538293 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.538266 2576 generic.go:356] "Generic (PLEG): container finished" podID="02732677-4194-48e6-9dca-e6cc3cc6bbe6" containerID="1809a0b2568fa06b9eb43ed1505934b781a28cf52252a5da24d6d4e46e54f30a" exitCode=0 Jul 21 21:24:51.538411 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.538356 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" event={"ID":"02732677-4194-48e6-9dca-e6cc3cc6bbe6","Type":"ContainerDied","Data":"1809a0b2568fa06b9eb43ed1505934b781a28cf52252a5da24d6d4e46e54f30a"} Jul 21 21:24:51.556654 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.556613 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:24:51.659738 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.659716 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-model-cache\") pod \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " Jul 21 21:24:51.659877 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.659743 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-home\") pod \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " Jul 21 21:24:51.659877 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.659807 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tls-certs\") pod \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " Jul 21 21:24:51.659877 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.659847 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzxhb\" (UniqueName: \"kubernetes.io/projected/02732677-4194-48e6-9dca-e6cc3cc6bbe6-kube-api-access-rzxhb\") pod \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " Jul 21 21:24:51.659877 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.659865 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-dshm\") pod \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " Jul 21 21:24:51.660095 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.659908 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tmp-dir\") pod \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\" (UID: \"02732677-4194-48e6-9dca-e6cc3cc6bbe6\") " Jul 21 21:24:51.660095 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.660020 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-model-cache" (OuterVolumeSpecName: "model-cache") pod "02732677-4194-48e6-9dca-e6cc3cc6bbe6" (UID: "02732677-4194-48e6-9dca-e6cc3cc6bbe6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:51.660095 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.660061 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-home" (OuterVolumeSpecName: "home") pod "02732677-4194-48e6-9dca-e6cc3cc6bbe6" (UID: "02732677-4194-48e6-9dca-e6cc3cc6bbe6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:51.660532 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.660231 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02732677-4194-48e6-9dca-e6cc3cc6bbe6" (UID: "02732677-4194-48e6-9dca-e6cc3cc6bbe6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:51.660532 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.660242 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-model-cache\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:24:51.660532 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.660259 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-home\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:24:51.661927 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.661901 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02732677-4194-48e6-9dca-e6cc3cc6bbe6-kube-api-access-rzxhb" (OuterVolumeSpecName: "kube-api-access-rzxhb") pod "02732677-4194-48e6-9dca-e6cc3cc6bbe6" (UID: "02732677-4194-48e6-9dca-e6cc3cc6bbe6"). InnerVolumeSpecName "kube-api-access-rzxhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:24:51.661927 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.661923 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02732677-4194-48e6-9dca-e6cc3cc6bbe6" (UID: "02732677-4194-48e6-9dca-e6cc3cc6bbe6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 21:24:51.662069 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.662003 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-dshm" (OuterVolumeSpecName: "dshm") pod "02732677-4194-48e6-9dca-e6cc3cc6bbe6" (UID: "02732677-4194-48e6-9dca-e6cc3cc6bbe6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 21:24:51.761677 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.761650 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tmp-dir\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:24:51.761677 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.761673 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02732677-4194-48e6-9dca-e6cc3cc6bbe6-tls-certs\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:24:51.761817 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.761683 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzxhb\" (UniqueName: \"kubernetes.io/projected/02732677-4194-48e6-9dca-e6cc3cc6bbe6-kube-api-access-rzxhb\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:24:51.761817 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:51.761692 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02732677-4194-48e6-9dca-e6cc3cc6bbe6-dshm\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:24:52.544141 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:52.544113 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" Jul 21 21:24:52.544641 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:52.544150 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b" event={"ID":"02732677-4194-48e6-9dca-e6cc3cc6bbe6","Type":"ContainerDied","Data":"246bf93f8adc5ccd8bd44887ac0557ce121ee4d32eaf7262983ca593dc9e5184"} Jul 21 21:24:52.544641 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:52.544194 2576 scope.go:117] "RemoveContainer" containerID="1809a0b2568fa06b9eb43ed1505934b781a28cf52252a5da24d6d4e46e54f30a" Jul 21 21:24:52.560029 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:52.560008 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b"] Jul 21 21:24:52.561754 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:52.561736 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-80e9ec/sub-test-model-80e9ec-kserve-f74f9f9c9-vnn6b"] Jul 21 21:24:53.708573 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:24:53.708509 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02732677-4194-48e6-9dca-e6cc3cc6bbe6" path="/var/lib/kubelet/pods/02732677-4194-48e6-9dca-e6cc3cc6bbe6/volumes" Jul 21 21:25:52.425479 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.425449 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-140225616918960"] Jul 21 21:25:52.425908 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.425803 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02732677-4194-48e6-9dca-e6cc3cc6bbe6" containerName="main" Jul 21 21:25:52.425908 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.425819 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02732677-4194-48e6-9dca-e6cc3cc6bbe6" containerName="main" Jul 21 21:25:52.425908 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.425895 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02732677-4194-48e6-9dca-e6cc3cc6bbe6" containerName="main" Jul 21 21:25:52.428732 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.428708 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:52.430990 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.430972 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m67qh\"" Jul 21 21:25:52.435397 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.435368 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225616918960"] Jul 21 21:25:52.529183 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.529162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrn54\" (UniqueName: \"kubernetes.io/projected/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752-kube-api-access-nrn54\") pod \"test-curl-7801-140225616918960\" (UID: \"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752\") " pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:52.630480 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.630454 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrn54\" (UniqueName: \"kubernetes.io/projected/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752-kube-api-access-nrn54\") pod \"test-curl-7801-140225616918960\" (UID: \"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752\") " pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:52.639169 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.639151 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrn54\" (UniqueName: \"kubernetes.io/projected/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752-kube-api-access-nrn54\") pod \"test-curl-7801-140225616918960\" (UID: \"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752\") " pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:52.738809 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.738786 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:52.865412 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.865388 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225616918960"] Jul 21 21:25:52.867588 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:25:52.867560 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e7b7983_ecc0_45e6_bbc7_3a4e8230d752.slice/crio-5790c48788e91fae4e2724e1a06a90f4506c3a2a6077f5c2c0a601bc4878c089 WatchSource:0}: Error finding container 5790c48788e91fae4e2724e1a06a90f4506c3a2a6077f5c2c0a601bc4878c089: Status 404 returned error can't find the container with id 5790c48788e91fae4e2724e1a06a90f4506c3a2a6077f5c2c0a601bc4878c089 Jul 21 21:25:52.868815 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:52.868793 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 21:25:53.826781 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:53.826724 2576 generic.go:356] "Generic (PLEG): container finished" podID="8e7b7983-ecc0-45e6-bbc7-3a4e8230d752" containerID="6747a25e57363001ff4224656481557f20b184a16a7b31ad0bd8fa426619044f" exitCode=0 Jul 21 21:25:53.827060 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:53.826777 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225616918960" event={"ID":"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752","Type":"ContainerDied","Data":"6747a25e57363001ff4224656481557f20b184a16a7b31ad0bd8fa426619044f"} Jul 21 21:25:53.827060 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:53.826812 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225616918960" event={"ID":"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752","Type":"ContainerStarted","Data":"5790c48788e91fae4e2724e1a06a90f4506c3a2a6077f5c2c0a601bc4878c089"} Jul 21 21:25:54.961108 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:54.961083 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:54.999051 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:54.999029 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7801-140225616918960_8e7b7983-ecc0-45e6-bbc7-3a4e8230d752/test-curl-7801-140225616918960/0.log" Jul 21 21:25:55.026579 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.026552 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-140225616918960"] Jul 21 21:25:55.030284 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.030261 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-140225616918960"] Jul 21 21:25:55.048277 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.048253 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrn54\" (UniqueName: \"kubernetes.io/projected/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752-kube-api-access-nrn54\") pod \"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752\" (UID: \"8e7b7983-ecc0-45e6-bbc7-3a4e8230d752\") " Jul 21 21:25:55.050417 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.050392 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752-kube-api-access-nrn54" (OuterVolumeSpecName: "kube-api-access-nrn54") pod "8e7b7983-ecc0-45e6-bbc7-3a4e8230d752" (UID: "8e7b7983-ecc0-45e6-bbc7-3a4e8230d752"). InnerVolumeSpecName "kube-api-access-nrn54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:25:55.149369 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.149308 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrn54\" (UniqueName: \"kubernetes.io/projected/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752-kube-api-access-nrn54\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:25:55.159114 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.159094 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-140225616916784"] Jul 21 21:25:55.159466 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.159453 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7b7983-ecc0-45e6-bbc7-3a4e8230d752" containerName="test-curl-7801-140225616918960" Jul 21 21:25:55.159511 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.159468 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7b7983-ecc0-45e6-bbc7-3a4e8230d752" containerName="test-curl-7801-140225616918960" Jul 21 21:25:55.159553 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.159543 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7b7983-ecc0-45e6-bbc7-3a4e8230d752" containerName="test-curl-7801-140225616918960" Jul 21 21:25:55.162352 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.162339 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:55.172347 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.172317 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225616916784"] Jul 21 21:25:55.250334 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.250300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t4s8\" (UniqueName: \"kubernetes.io/projected/197c7159-1fc8-4323-a600-c43a0fd1a5df-kube-api-access-6t4s8\") pod \"test-curl-7801-140225616916784\" (UID: \"197c7159-1fc8-4323-a600-c43a0fd1a5df\") " pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:55.351268 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.351236 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t4s8\" (UniqueName: \"kubernetes.io/projected/197c7159-1fc8-4323-a600-c43a0fd1a5df-kube-api-access-6t4s8\") pod \"test-curl-7801-140225616916784\" (UID: \"197c7159-1fc8-4323-a600-c43a0fd1a5df\") " pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:55.360703 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.360677 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t4s8\" (UniqueName: \"kubernetes.io/projected/197c7159-1fc8-4323-a600-c43a0fd1a5df-kube-api-access-6t4s8\") pod \"test-curl-7801-140225616916784\" (UID: \"197c7159-1fc8-4323-a600-c43a0fd1a5df\") " pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:55.480273 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.480249 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:55.606358 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.606334 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225616916784"] Jul 21 21:25:55.608652 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:25:55.608629 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod197c7159_1fc8_4323_a600_c43a0fd1a5df.slice/crio-c63414024abe31e8170a6fa0781c170838d4a7d24c40b6a10a470d15170a465e WatchSource:0}: Error finding container c63414024abe31e8170a6fa0781c170838d4a7d24c40b6a10a470d15170a465e: Status 404 returned error can't find the container with id c63414024abe31e8170a6fa0781c170838d4a7d24c40b6a10a470d15170a465e Jul 21 21:25:55.708253 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.708226 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e7b7983-ecc0-45e6-bbc7-3a4e8230d752" path="/var/lib/kubelet/pods/8e7b7983-ecc0-45e6-bbc7-3a4e8230d752/volumes" Jul 21 21:25:55.836641 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.836591 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225616916784" event={"ID":"197c7159-1fc8-4323-a600-c43a0fd1a5df","Type":"ContainerStarted","Data":"c63414024abe31e8170a6fa0781c170838d4a7d24c40b6a10a470d15170a465e"} Jul 21 21:25:55.837816 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.837800 2576 scope.go:117] "RemoveContainer" containerID="6747a25e57363001ff4224656481557f20b184a16a7b31ad0bd8fa426619044f" Jul 21 21:25:55.837893 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:55.837818 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616918960" Jul 21 21:25:56.844665 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:56.844634 2576 generic.go:356] "Generic (PLEG): container finished" podID="197c7159-1fc8-4323-a600-c43a0fd1a5df" containerID="dc1a961ad6d511dea6cac5f5d2fa52e6d942b9fc5e094f2d0bb698a1325e29cd" exitCode=0 Jul 21 21:25:56.845019 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:56.844673 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225616916784" event={"ID":"197c7159-1fc8-4323-a600-c43a0fd1a5df","Type":"ContainerDied","Data":"dc1a961ad6d511dea6cac5f5d2fa52e6d942b9fc5e094f2d0bb698a1325e29cd"} Jul 21 21:25:57.982820 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:57.982799 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:58.004586 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.004564 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7801-140225616916784_197c7159-1fc8-4323-a600-c43a0fd1a5df/test-curl-7801-140225616916784/0.log" Jul 21 21:25:58.028678 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.028651 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-140225616916784"] Jul 21 21:25:58.030697 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.030678 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-140225616916784"] Jul 21 21:25:58.075741 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.075719 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t4s8\" (UniqueName: \"kubernetes.io/projected/197c7159-1fc8-4323-a600-c43a0fd1a5df-kube-api-access-6t4s8\") pod \"197c7159-1fc8-4323-a600-c43a0fd1a5df\" (UID: \"197c7159-1fc8-4323-a600-c43a0fd1a5df\") " Jul 21 21:25:58.077684 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.077659 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/197c7159-1fc8-4323-a600-c43a0fd1a5df-kube-api-access-6t4s8" (OuterVolumeSpecName: "kube-api-access-6t4s8") pod "197c7159-1fc8-4323-a600-c43a0fd1a5df" (UID: "197c7159-1fc8-4323-a600-c43a0fd1a5df"). InnerVolumeSpecName "kube-api-access-6t4s8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:25:58.176719 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.176663 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t4s8\" (UniqueName: \"kubernetes.io/projected/197c7159-1fc8-4323-a600-c43a0fd1a5df-kube-api-access-6t4s8\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:25:58.191791 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.191764 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:25:58.192122 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.192110 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="197c7159-1fc8-4323-a600-c43a0fd1a5df" containerName="test-curl-7801-140225616916784" Jul 21 21:25:58.192166 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.192125 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="197c7159-1fc8-4323-a600-c43a0fd1a5df" containerName="test-curl-7801-140225616916784" Jul 21 21:25:58.192202 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.192191 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="197c7159-1fc8-4323-a600-c43a0fd1a5df" containerName="test-curl-7801-140225616916784" Jul 21 21:25:58.194781 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.194764 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:25:58.199844 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.199824 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:25:58.277589 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.277567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvbj4\" (UniqueName: \"kubernetes.io/projected/afb80edc-bfe1-4326-844c-f166b8a344e5-kube-api-access-rvbj4\") pod \"test-curl-7801-140225617060896\" (UID: \"afb80edc-bfe1-4326-844c-f166b8a344e5\") " pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:25:58.378604 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.378575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvbj4\" (UniqueName: \"kubernetes.io/projected/afb80edc-bfe1-4326-844c-f166b8a344e5-kube-api-access-rvbj4\") pod \"test-curl-7801-140225617060896\" (UID: \"afb80edc-bfe1-4326-844c-f166b8a344e5\") " pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:25:58.388194 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.388169 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvbj4\" (UniqueName: \"kubernetes.io/projected/afb80edc-bfe1-4326-844c-f166b8a344e5-kube-api-access-rvbj4\") pod \"test-curl-7801-140225617060896\" (UID: \"afb80edc-bfe1-4326-844c-f166b8a344e5\") " pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:25:58.505700 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.505683 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:25:58.632941 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.632916 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:25:58.634793 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:25:58.634762 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafb80edc_bfe1_4326_844c_f166b8a344e5.slice/crio-8813d98bd6353241f2043919e9241a5b771be2b3ea04a6c4406ee7673ee5c20f WatchSource:0}: Error finding container 8813d98bd6353241f2043919e9241a5b771be2b3ea04a6c4406ee7673ee5c20f: Status 404 returned error can't find the container with id 8813d98bd6353241f2043919e9241a5b771be2b3ea04a6c4406ee7673ee5c20f Jul 21 21:25:58.854992 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.854933 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225617060896" event={"ID":"afb80edc-bfe1-4326-844c-f166b8a344e5","Type":"ContainerStarted","Data":"8813d98bd6353241f2043919e9241a5b771be2b3ea04a6c4406ee7673ee5c20f"} Jul 21 21:25:58.856086 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.856071 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c63414024abe31e8170a6fa0781c170838d4a7d24c40b6a10a470d15170a465e" Jul 21 21:25:58.856153 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:58.856093 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:58.866424 ip-10-0-128-226 kubenswrapper[2576]: E0721 21:25:58.866403 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7801-140225616916784\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" podUID="197c7159-1fc8-4323-a600-c43a0fd1a5df" pod="opendatahub/test-curl-7801-140225616916784" Jul 21 21:25:59.708301 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:59.708273 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="197c7159-1fc8-4323-a600-c43a0fd1a5df" path="/var/lib/kubelet/pods/197c7159-1fc8-4323-a600-c43a0fd1a5df/volumes" Jul 21 21:25:59.867371 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:59.867336 2576 generic.go:356] "Generic (PLEG): container finished" podID="afb80edc-bfe1-4326-844c-f166b8a344e5" containerID="c67468059aeb0851214aecdbe15ca1f8603c9d8f16bc512a3cfdb6fad293a383" exitCode=0 Jul 21 21:25:59.867537 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:25:59.867389 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225617060896" event={"ID":"afb80edc-bfe1-4326-844c-f166b8a344e5","Type":"ContainerDied","Data":"c67468059aeb0851214aecdbe15ca1f8603c9d8f16bc512a3cfdb6fad293a383"} Jul 21 21:26:01.006962 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.006938 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.027864 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.027843 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7801-140225617060896_afb80edc-bfe1-4326-844c-f166b8a344e5/test-curl-7801-140225617060896/0.log" Jul 21 21:26:01.049680 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.049650 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:01.053160 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.053137 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:01.098817 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.098794 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvbj4\" (UniqueName: \"kubernetes.io/projected/afb80edc-bfe1-4326-844c-f166b8a344e5-kube-api-access-rvbj4\") pod \"afb80edc-bfe1-4326-844c-f166b8a344e5\" (UID: \"afb80edc-bfe1-4326-844c-f166b8a344e5\") " Jul 21 21:26:01.100843 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.100819 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb80edc-bfe1-4326-844c-f166b8a344e5-kube-api-access-rvbj4" (OuterVolumeSpecName: "kube-api-access-rvbj4") pod "afb80edc-bfe1-4326-844c-f166b8a344e5" (UID: "afb80edc-bfe1-4326-844c-f166b8a344e5"). InnerVolumeSpecName "kube-api-access-rvbj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:26:01.176661 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.176597 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:01.177015 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.176997 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb80edc-bfe1-4326-844c-f166b8a344e5" containerName="test-curl-7801-140225617060896" Jul 21 21:26:01.177087 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.177018 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb80edc-bfe1-4326-844c-f166b8a344e5" containerName="test-curl-7801-140225617060896" Jul 21 21:26:01.177150 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.177137 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="afb80edc-bfe1-4326-844c-f166b8a344e5" containerName="test-curl-7801-140225617060896" Jul 21 21:26:01.187581 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.187564 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:01.187681 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.187653 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.200050 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.200028 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvbj4\" (UniqueName: \"kubernetes.io/projected/afb80edc-bfe1-4326-844c-f166b8a344e5-kube-api-access-rvbj4\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:26:01.300733 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.300703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twzbg\" (UniqueName: \"kubernetes.io/projected/1888a822-f9be-4ff4-8963-5a88aa213766-kube-api-access-twzbg\") pod \"test-curl-7801-140225617060896\" (UID: \"1888a822-f9be-4ff4-8963-5a88aa213766\") " pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.401531 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.401499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twzbg\" (UniqueName: \"kubernetes.io/projected/1888a822-f9be-4ff4-8963-5a88aa213766-kube-api-access-twzbg\") pod \"test-curl-7801-140225617060896\" (UID: \"1888a822-f9be-4ff4-8963-5a88aa213766\") " pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.410758 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.410732 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twzbg\" (UniqueName: \"kubernetes.io/projected/1888a822-f9be-4ff4-8963-5a88aa213766-kube-api-access-twzbg\") pod \"test-curl-7801-140225617060896\" (UID: \"1888a822-f9be-4ff4-8963-5a88aa213766\") " pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.496822 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.496798 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.623793 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.623771 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:01.625419 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:26:01.625396 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1888a822_f9be_4ff4_8963_5a88aa213766.slice/crio-e1993249b63b952cd8d2449f6ccd751472d3ab9b5281bec685f8685d19afbc73 WatchSource:0}: Error finding container e1993249b63b952cd8d2449f6ccd751472d3ab9b5281bec685f8685d19afbc73: Status 404 returned error can't find the container with id e1993249b63b952cd8d2449f6ccd751472d3ab9b5281bec685f8685d19afbc73 Jul 21 21:26:01.709663 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.709639 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb80edc-bfe1-4326-844c-f166b8a344e5" path="/var/lib/kubelet/pods/afb80edc-bfe1-4326-844c-f166b8a344e5/volumes" Jul 21 21:26:01.877479 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.877456 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:01.877591 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.877498 2576 scope.go:117] "RemoveContainer" containerID="c67468059aeb0851214aecdbe15ca1f8603c9d8f16bc512a3cfdb6fad293a383" Jul 21 21:26:01.878784 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:01.878759 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225617060896" event={"ID":"1888a822-f9be-4ff4-8963-5a88aa213766","Type":"ContainerStarted","Data":"e1993249b63b952cd8d2449f6ccd751472d3ab9b5281bec685f8685d19afbc73"} Jul 21 21:26:02.885556 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:02.885526 2576 generic.go:356] "Generic (PLEG): container finished" podID="1888a822-f9be-4ff4-8963-5a88aa213766" containerID="7d530f0431720fef39f3a8a315cd3d534f3b621971e79ba72469fbd6a863384a" exitCode=0 Jul 21 21:26:02.885934 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:02.885623 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225617060896" event={"ID":"1888a822-f9be-4ff4-8963-5a88aa213766","Type":"ContainerDied","Data":"7d530f0431720fef39f3a8a315cd3d534f3b621971e79ba72469fbd6a863384a"} Jul 21 21:26:04.018877 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.018856 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:04.039396 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.039361 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7801-140225617060896_1888a822-f9be-4ff4-8963-5a88aa213766/test-curl-7801-140225617060896/0.log" Jul 21 21:26:04.070804 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.070771 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:04.070924 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.070831 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-140225617060896"] Jul 21 21:26:04.124369 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.124345 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twzbg\" (UniqueName: \"kubernetes.io/projected/1888a822-f9be-4ff4-8963-5a88aa213766-kube-api-access-twzbg\") pod \"1888a822-f9be-4ff4-8963-5a88aa213766\" (UID: \"1888a822-f9be-4ff4-8963-5a88aa213766\") " Jul 21 21:26:04.126302 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.126277 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1888a822-f9be-4ff4-8963-5a88aa213766-kube-api-access-twzbg" (OuterVolumeSpecName: "kube-api-access-twzbg") pod "1888a822-f9be-4ff4-8963-5a88aa213766" (UID: "1888a822-f9be-4ff4-8963-5a88aa213766"). InnerVolumeSpecName "kube-api-access-twzbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:26:04.195459 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.195392 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7801-140225617061904"] Jul 21 21:26:04.195776 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.195763 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1888a822-f9be-4ff4-8963-5a88aa213766" containerName="test-curl-7801-140225617060896" Jul 21 21:26:04.195817 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.195778 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1888a822-f9be-4ff4-8963-5a88aa213766" containerName="test-curl-7801-140225617060896" Jul 21 21:26:04.195854 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.195831 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1888a822-f9be-4ff4-8963-5a88aa213766" containerName="test-curl-7801-140225617060896" Jul 21 21:26:04.198638 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.198624 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:04.206461 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.206420 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225617061904"] Jul 21 21:26:04.225386 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.225363 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twzbg\" (UniqueName: \"kubernetes.io/projected/1888a822-f9be-4ff4-8963-5a88aa213766-kube-api-access-twzbg\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:26:04.326651 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.326629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccpt8\" (UniqueName: \"kubernetes.io/projected/eabecd73-2622-4294-99b6-5a039ef9d848-kube-api-access-ccpt8\") pod \"test-curl-7801-140225617061904\" (UID: \"eabecd73-2622-4294-99b6-5a039ef9d848\") " pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:04.427699 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.427671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccpt8\" (UniqueName: \"kubernetes.io/projected/eabecd73-2622-4294-99b6-5a039ef9d848-kube-api-access-ccpt8\") pod \"test-curl-7801-140225617061904\" (UID: \"eabecd73-2622-4294-99b6-5a039ef9d848\") " pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:04.435912 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.435894 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccpt8\" (UniqueName: \"kubernetes.io/projected/eabecd73-2622-4294-99b6-5a039ef9d848-kube-api-access-ccpt8\") pod \"test-curl-7801-140225617061904\" (UID: \"eabecd73-2622-4294-99b6-5a039ef9d848\") " pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:04.508555 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.508537 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:04.634117 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.634070 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7801-140225617061904"] Jul 21 21:26:04.636334 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:26:04.636306 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeabecd73_2622_4294_99b6_5a039ef9d848.slice/crio-04bb8ef0503e70ef0c1646fe52aa367e5a14015956ca19ec7b9f172fb6104207 WatchSource:0}: Error finding container 04bb8ef0503e70ef0c1646fe52aa367e5a14015956ca19ec7b9f172fb6104207: Status 404 returned error can't find the container with id 04bb8ef0503e70ef0c1646fe52aa367e5a14015956ca19ec7b9f172fb6104207 Jul 21 21:26:04.895705 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.895649 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:04.895705 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.895677 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1993249b63b952cd8d2449f6ccd751472d3ab9b5281bec685f8685d19afbc73" Jul 21 21:26:04.896811 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:04.896786 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225617061904" event={"ID":"eabecd73-2622-4294-99b6-5a039ef9d848","Type":"ContainerStarted","Data":"04bb8ef0503e70ef0c1646fe52aa367e5a14015956ca19ec7b9f172fb6104207"} Jul 21 21:26:04.906603 ip-10-0-128-226 kubenswrapper[2576]: E0721 21:26:04.906575 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7801-140225617060896\" is forbidden: User \"system:node:ip-10-0-128-226.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-226.ec2.internal' and this object" podUID="1888a822-f9be-4ff4-8963-5a88aa213766" pod="opendatahub/test-curl-7801-140225617060896" Jul 21 21:26:05.708613 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:05.708585 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1888a822-f9be-4ff4-8963-5a88aa213766" path="/var/lib/kubelet/pods/1888a822-f9be-4ff4-8963-5a88aa213766/volumes" Jul 21 21:26:05.903067 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:05.903037 2576 generic.go:356] "Generic (PLEG): container finished" podID="eabecd73-2622-4294-99b6-5a039ef9d848" containerID="75576b0ee36c222eba3939794868ddd2cb54a2c885efd684f6fe3d0e532f0bb4" exitCode=0 Jul 21 21:26:05.903194 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:05.903110 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7801-140225617061904" event={"ID":"eabecd73-2622-4294-99b6-5a039ef9d848","Type":"ContainerDied","Data":"75576b0ee36c222eba3939794868ddd2cb54a2c885efd684f6fe3d0e532f0bb4"} Jul 21 21:26:07.040944 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.040925 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:07.061948 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.061907 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7801-140225617061904_eabecd73-2622-4294-99b6-5a039ef9d848/test-curl-7801-140225617061904/0.log" Jul 21 21:26:07.087653 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.087621 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7801-140225617061904"] Jul 21 21:26:07.089248 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.089230 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7801-140225617061904"] Jul 21 21:26:07.149585 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.149558 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccpt8\" (UniqueName: \"kubernetes.io/projected/eabecd73-2622-4294-99b6-5a039ef9d848-kube-api-access-ccpt8\") pod \"eabecd73-2622-4294-99b6-5a039ef9d848\" (UID: \"eabecd73-2622-4294-99b6-5a039ef9d848\") " Jul 21 21:26:07.151606 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.151584 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eabecd73-2622-4294-99b6-5a039ef9d848-kube-api-access-ccpt8" (OuterVolumeSpecName: "kube-api-access-ccpt8") pod "eabecd73-2622-4294-99b6-5a039ef9d848" (UID: "eabecd73-2622-4294-99b6-5a039ef9d848"). InnerVolumeSpecName "kube-api-access-ccpt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 21:26:07.250493 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.250468 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccpt8\" (UniqueName: \"kubernetes.io/projected/eabecd73-2622-4294-99b6-5a039ef9d848-kube-api-access-ccpt8\") on node \"ip-10-0-128-226.ec2.internal\" DevicePath \"\"" Jul 21 21:26:07.711949 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.711925 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eabecd73-2622-4294-99b6-5a039ef9d848" path="/var/lib/kubelet/pods/eabecd73-2622-4294-99b6-5a039ef9d848/volumes" Jul 21 21:26:07.912568 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.912547 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7801-140225617061904" Jul 21 21:26:07.912731 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:07.912604 2576 scope.go:117] "RemoveContainer" containerID="75576b0ee36c222eba3939794868ddd2cb54a2c885efd684f6fe3d0e532f0bb4" Jul 21 21:26:55.183858 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:26:55.183827 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5658b9f6d7-d96qt_f44f0265-89da-4e7d-8afd-e42618681cd1/authorino/0.log" Jul 21 21:27:01.580641 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:01.580606 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v2dtx_9e205eb3-a779-478c-8069-82f621d3cb7b/server/0.log" Jul 21 21:27:01.708312 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:01.708285 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7xpb6_b6de02f7-c3a7-48c1-9f6b-cc99fd54f096/manager/0.log" Jul 21 21:27:01.962465 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:01.962425 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-68hjr_94b99f44-1287-4604-9a75-063a178fa4e4/copy-manifests/0.log" Jul 21 21:27:01.978476 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:01.978453 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-68hjr_94b99f44-1287-4604-9a75-063a178fa4e4/manager/0.log" Jul 21 21:27:04.298725 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:04.298699 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5658b9f6d7-d96qt_f44f0265-89da-4e7d-8afd-e42618681cd1/authorino/0.log" Jul 21 21:27:04.669839 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:04.669779 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9q6x5_6b461c54-a82c-4396-a0c9-f9b60b104634/kuadrant-console-plugin/0.log" Jul 21 21:27:05.815901 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:05.815851 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-6pzgw_dd6189de-f4ca-49b2-8c07-3cbd3942f6f5/kube-auth-proxy/0.log" Jul 21 21:27:06.306356 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:06.306324 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-qmcc6_5e14f08e-dd79-47ed-9676-ffddd218ad93/payload-processing/0.log" Jul 21 21:27:06.934449 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:06.934416 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zs656_b703768a-6c96-4939-9257-d788324e5dc0/main/0.log" Jul 21 21:27:07.305327 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:07.305295 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nr4xz_eb145dfb-89b8-48b7-85b6-321593d65f9a/main/0.log" Jul 21 21:27:07.417783 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:07.417765 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-rltgt_707c724d-923e-4a4e-ad6f-f6d846ca3f42/main/0.log" Jul 21 21:27:16.581498 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:16.581469 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5ttxj_139d15ea-a86c-4eb5-8cdd-de80d79de8af/global-pull-secret-syncer/0.log" Jul 21 21:27:16.741823 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:16.741796 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dpg8j_9eb63ef7-b4d9-4cd0-a8f2-68716975488b/konnectivity-agent/0.log" Jul 21 21:27:16.806093 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:16.806066 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-226.ec2.internal_470709dad7cfb08f110d2e7d10c32b8e/haproxy/0.log" Jul 21 21:27:21.117776 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:21.117738 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5658b9f6d7-d96qt_f44f0265-89da-4e7d-8afd-e42618681cd1/authorino/0.log" Jul 21 21:27:21.220599 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:21.220578 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9q6x5_6b461c54-a82c-4396-a0c9-f9b60b104634/kuadrant-console-plugin/0.log" Jul 21 21:27:23.232613 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.232587 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/alertmanager/0.log" Jul 21 21:27:23.253742 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.253717 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/config-reloader/0.log" Jul 21 21:27:23.275066 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.275044 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/kube-rbac-proxy-web/0.log" Jul 21 21:27:23.296270 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.296248 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/kube-rbac-proxy/0.log" Jul 21 21:27:23.318927 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.318909 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/kube-rbac-proxy-metric/0.log" Jul 21 21:27:23.343097 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.343078 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/prom-label-proxy/0.log" Jul 21 21:27:23.364161 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.364143 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1e3a63c7-1fa2-460d-a186-1f649e81d662/init-config-reloader/0.log" Jul 21 21:27:23.669155 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.669084 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bchc5_a1efedd5-d848-44af-bacc-5e2ce0e56a4e/node-exporter/0.log" Jul 21 21:27:23.690326 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.690307 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bchc5_a1efedd5-d848-44af-bacc-5e2ce0e56a4e/kube-rbac-proxy/0.log" Jul 21 21:27:23.717062 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.717039 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bchc5_a1efedd5-d848-44af-bacc-5e2ce0e56a4e/init-textfile/0.log" Jul 21 21:27:23.855912 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.855887 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-7sqrp_b7cefcae-d372-4dd8-9ced-d12b33cbce11/kube-rbac-proxy-main/0.log" Jul 21 21:27:23.891248 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.891220 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-7sqrp_b7cefcae-d372-4dd8-9ced-d12b33cbce11/kube-rbac-proxy-self/0.log" Jul 21 21:27:23.912746 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:23.912728 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-7sqrp_b7cefcae-d372-4dd8-9ced-d12b33cbce11/openshift-state-metrics/0.log" Jul 21 21:27:24.356567 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.356539 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bbcbb5f86-s6l96_a4865f26-dc8e-4787-b54f-e99309141c9d/telemeter-client/0.log" Jul 21 21:27:24.379538 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.379515 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bbcbb5f86-s6l96_a4865f26-dc8e-4787-b54f-e99309141c9d/reload/0.log" Jul 21 21:27:24.401468 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.401429 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bbcbb5f86-s6l96_a4865f26-dc8e-4787-b54f-e99309141c9d/kube-rbac-proxy/0.log" Jul 21 21:27:24.433177 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.433158 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d6b796b-97rj4_4b9d673e-952b-4fc3-9d4d-f152748281ca/thanos-query/0.log" Jul 21 21:27:24.454198 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.454180 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d6b796b-97rj4_4b9d673e-952b-4fc3-9d4d-f152748281ca/kube-rbac-proxy-web/0.log" Jul 21 21:27:24.486923 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.486905 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d6b796b-97rj4_4b9d673e-952b-4fc3-9d4d-f152748281ca/kube-rbac-proxy/0.log" Jul 21 21:27:24.511448 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.511414 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d6b796b-97rj4_4b9d673e-952b-4fc3-9d4d-f152748281ca/prom-label-proxy/0.log" Jul 21 21:27:24.535087 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.535074 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d6b796b-97rj4_4b9d673e-952b-4fc3-9d4d-f152748281ca/kube-rbac-proxy-rules/0.log" Jul 21 21:27:24.560902 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:24.560884 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b4d6b796b-97rj4_4b9d673e-952b-4fc3-9d4d-f152748281ca/kube-rbac-proxy-metrics/0.log" Jul 21 21:27:25.864193 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:25.864170 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-vjqqx_927de61a-16ae-45d1-8b5b-4f03cc1c6af9/networking-console-plugin/0.log" Jul 21 21:27:27.486863 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.486837 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj"] Jul 21 21:27:27.487233 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.487162 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eabecd73-2622-4294-99b6-5a039ef9d848" containerName="test-curl-7801-140225617061904" Jul 21 21:27:27.487233 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.487174 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eabecd73-2622-4294-99b6-5a039ef9d848" containerName="test-curl-7801-140225617061904" Jul 21 21:27:27.487308 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.487244 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eabecd73-2622-4294-99b6-5a039ef9d848" containerName="test-curl-7801-140225617061904" Jul 21 21:27:27.490257 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.490238 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.492592 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.492571 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-75fnx\"/\"default-dockercfg-6ncg2\"" Jul 21 21:27:27.492771 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.492758 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-75fnx\"/\"kube-root-ca.crt\"" Jul 21 21:27:27.493418 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.493398 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-75fnx\"/\"openshift-service-ca.crt\"" Jul 21 21:27:27.500124 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.500106 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj"] Jul 21 21:27:27.647360 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.647337 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7cvn\" (UniqueName: \"kubernetes.io/projected/af25bf26-18f1-4a2a-bf13-ffbb09700b59-kube-api-access-m7cvn\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.647498 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.647384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-proc\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.647498 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.647413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-lib-modules\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.647498 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.647470 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-podres\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.647498 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.647498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-sys\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748298 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748238 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7cvn\" (UniqueName: \"kubernetes.io/projected/af25bf26-18f1-4a2a-bf13-ffbb09700b59-kube-api-access-m7cvn\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748298 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-proc\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748431 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748310 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-lib-modules\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748431 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-podres\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748431 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748398 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-sys\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748431 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-lib-modules\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748590 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748458 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-proc\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748590 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748496 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-podres\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.748590 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.748518 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af25bf26-18f1-4a2a-bf13-ffbb09700b59-sys\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.756780 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.756740 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7cvn\" (UniqueName: \"kubernetes.io/projected/af25bf26-18f1-4a2a-bf13-ffbb09700b59-kube-api-access-m7cvn\") pod \"perf-node-gather-daemonset-59nrj\" (UID: \"af25bf26-18f1-4a2a-bf13-ffbb09700b59\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.800003 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.799982 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:27.929967 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:27.929943 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj"] Jul 21 21:27:27.931501 ip-10-0-128-226 kubenswrapper[2576]: W0721 21:27:27.931473 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaf25bf26_18f1_4a2a_bf13_ffbb09700b59.slice/crio-296264dcf98065709e2106a5d491aaaccc75f43e233baf536b5a7113680c7934 WatchSource:0}: Error finding container 296264dcf98065709e2106a5d491aaaccc75f43e233baf536b5a7113680c7934: Status 404 returned error can't find the container with id 296264dcf98065709e2106a5d491aaaccc75f43e233baf536b5a7113680c7934 Jul 21 21:27:28.285170 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:28.285145 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" event={"ID":"af25bf26-18f1-4a2a-bf13-ffbb09700b59","Type":"ContainerStarted","Data":"547080d178942b1eef02ad1b2a9625fba2a781544112eace52e0394a4148b41c"} Jul 21 21:27:28.285293 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:28.285192 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" event={"ID":"af25bf26-18f1-4a2a-bf13-ffbb09700b59","Type":"ContainerStarted","Data":"296264dcf98065709e2106a5d491aaaccc75f43e233baf536b5a7113680c7934"} Jul 21 21:27:28.285293 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:28.285253 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:28.302290 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:28.302224 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" podStartSLOduration=1.3022117500000001 podStartE2EDuration="1.30221175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:27:28.301189764 +0000 UTC m=+2847.155260554" watchObservedRunningTime="2026-07-21 21:27:28.30221175 +0000 UTC m=+2847.156282477" Jul 21 21:27:29.089362 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:29.089326 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqj9x_22cdac8f-b180-4cc2-974d-20aba911febe/dns/0.log" Jul 21 21:27:29.111826 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:29.111805 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqj9x_22cdac8f-b180-4cc2-974d-20aba911febe/kube-rbac-proxy/0.log" Jul 21 21:27:29.166643 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:29.166611 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pxfnh_de325698-eb9b-450c-94e2-b938cfedff1b/dns-node-resolver/0.log" Jul 21 21:27:29.721669 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:29.721640 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-658585cf49-2lh2j_ac132a7b-664d-4d8c-803b-5a0ee6a00543/registry/0.log" Jul 21 21:27:29.793725 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:29.793701 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nflj4_286252e9-ce85-4212-9c4f-6dd0f139c56a/node-ca/0.log" Jul 21 21:27:30.867946 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:30.867922 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-6pzgw_dd6189de-f4ca-49b2-8c07-3cbd3942f6f5/kube-auth-proxy/0.log" Jul 21 21:27:31.015581 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:31.015555 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-qmcc6_5e14f08e-dd79-47ed-9676-ffddd218ad93/payload-processing/0.log" Jul 21 21:27:31.721665 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:31.721641 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-52g77_ad292553-bfa5-43e2-a1cb-a0ea99d2b93d/serve-healthcheck-canary/0.log" Jul 21 21:27:32.474704 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:32.474675 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qfqwd_7854f0d9-93f7-4038-bc67-a38ba102e9e1/kube-rbac-proxy/0.log" Jul 21 21:27:32.499819 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:32.499799 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qfqwd_7854f0d9-93f7-4038-bc67-a38ba102e9e1/exporter/0.log" Jul 21 21:27:32.529976 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:32.529955 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qfqwd_7854f0d9-93f7-4038-bc67-a38ba102e9e1/extractor/0.log" Jul 21 21:27:34.298938 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:34.298913 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-59nrj" Jul 21 21:27:34.938694 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:34.938662 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v2dtx_9e205eb3-a779-478c-8069-82f621d3cb7b/server/0.log" Jul 21 21:27:35.013493 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:35.013463 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7xpb6_b6de02f7-c3a7-48c1-9f6b-cc99fd54f096/manager/0.log" Jul 21 21:27:35.135145 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:35.135098 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-68hjr_94b99f44-1287-4604-9a75-063a178fa4e4/manager/0.log" Jul 21 21:27:35.154156 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:35.154133 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-68hjr_94b99f44-1287-4604-9a75-063a178fa4e4/copy-manifests/0.log" Jul 21 21:27:36.476121 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:36.476092 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66f9b7665f-9tjvb_1b6d6f46-07a5-4929-9dd7-e4ec3262b0b7/manager/0.log" Jul 21 21:27:37.318298 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:37.318273 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-6pzgw_dd6189de-f4ca-49b2-8c07-3cbd3942f6f5/kube-auth-proxy/0.log" Jul 21 21:27:37.427274 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:37.427218 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-qmcc6_5e14f08e-dd79-47ed-9676-ffddd218ad93/payload-processing/0.log" Jul 21 21:27:38.781355 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:38.781289 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-6pzgw_dd6189de-f4ca-49b2-8c07-3cbd3942f6f5/kube-auth-proxy/0.log" Jul 21 21:27:38.901402 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:38.901374 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-qmcc6_5e14f08e-dd79-47ed-9676-ffddd218ad93/payload-processing/0.log" Jul 21 21:27:43.252167 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.252141 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/kube-multus-additional-cni-plugins/0.log" Jul 21 21:27:43.272395 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.272374 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/egress-router-binary-copy/0.log" Jul 21 21:27:43.291962 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.291938 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/cni-plugins/0.log" Jul 21 21:27:43.313197 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.313178 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/bond-cni-plugin/0.log" Jul 21 21:27:43.333531 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.333512 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/routeoverride-cni/0.log" Jul 21 21:27:43.353856 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.353839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/whereabouts-cni-bincopy/0.log" Jul 21 21:27:43.379098 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.373839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2xmlh_4a2bd6b0-afa7-4ac2-8d11-959c15e8ee15/whereabouts-cni/0.log" Jul 21 21:27:43.715722 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.715682 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-llsxj_0903752f-002d-46ac-8c54-c1cfda27a347/kube-multus/0.log" Jul 21 21:27:43.815938 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.815913 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gdgtz_90b30272-adf6-4c30-b16a-8a950410acc8/network-metrics-daemon/0.log" Jul 21 21:27:43.839448 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:43.839407 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gdgtz_90b30272-adf6-4c30-b16a-8a950410acc8/kube-rbac-proxy/0.log" Jul 21 21:27:44.928336 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:44.928314 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/ovn-controller/0.log" Jul 21 21:27:44.948451 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:44.948408 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/ovn-acl-logging/0.log" Jul 21 21:27:44.970087 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:44.970068 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/kube-rbac-proxy-node/0.log" Jul 21 21:27:44.990243 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:44.990222 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/kube-rbac-proxy-ovn-metrics/0.log" Jul 21 21:27:45.011589 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:45.011569 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/northd/0.log" Jul 21 21:27:45.032511 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:45.032490 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/nbdb/0.log" Jul 21 21:27:45.055491 ip-10-0-128-226 kubenswrapper[2576]: I0721 21:27:45.055466 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g4gq6_0212f89e-a5b2-445c-97df-8d71dbf1cd76/sbdb/0.log"