Jul 08 17:31:30.836243 ip-10-0-128-115 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 17:31:30.836253 ip-10-0-128-115 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 17:31:30.836260 ip-10-0-128-115 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 17:31:30.836546 ip-10-0-128-115 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 17:31:40.893298 ip-10-0-128-115 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 17:31:40.893312 ip-10-0-128-115 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3e8e4756637640fa8321a505c92c589d -- Jul 08 17:33:47.384132 ip-10-0-128-115 systemd[1]: Starting Kubernetes Kubelet... Jul 08 17:33:47.797278 ip-10-0-128-115 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:33:47.797278 ip-10-0-128-115 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 17:33:47.797278 ip-10-0-128-115 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:33:47.797278 ip-10-0-128-115 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 17:33:47.797278 ip-10-0-128-115 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:33:47.798459 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.797296 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 17:33:47.801331 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801313 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:33:47.801331 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801332 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801337 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801340 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801343 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801346 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801349 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801351 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801354 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801357 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801359 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801362 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801364 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801368 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801371 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801373 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801376 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801378 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801382 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801385 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801387 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:33:47.801397 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801390 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801392 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801395 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801398 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801400 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801403 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801406 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801409 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801411 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801414 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801418 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801422 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801425 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801427 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801430 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801433 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801435 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801438 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801440 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:33:47.801903 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801443 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801446 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801448 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801451 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801453 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801455 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801459 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801462 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801465 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801468 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801470 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801473 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801475 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801478 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801481 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801484 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801487 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801490 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801493 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801495 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:33:47.802375 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801497 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801502 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801505 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801508 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801510 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801513 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801515 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801517 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801520 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801523 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801525 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801528 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801530 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801533 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801535 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801538 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801540 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801542 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801545 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:33:47.802897 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801547 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801550 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801552 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801554 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801557 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801559 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801562 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801564 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801567 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801570 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801573 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801575 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801578 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801580 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801583 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801586 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801588 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801590 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801593 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.801595 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:33:47.803367 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802294 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802301 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802303 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802307 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802310 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802313 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802315 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802318 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802320 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802323 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802325 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802327 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802333 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802337 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802339 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802343 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802347 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802350 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802353 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:33:47.803857 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802356 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802359 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802362 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802365 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802368 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802371 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802373 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802376 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802379 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802381 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802384 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802387 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802389 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802391 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802394 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802396 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802399 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802402 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802404 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802407 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:33:47.804341 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802409 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802412 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802414 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802417 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802419 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802423 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802425 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802428 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802432 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802435 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802437 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802440 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802442 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802446 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802448 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802450 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802453 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802456 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802458 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802461 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:33:47.804838 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802463 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802466 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802468 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802471 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802473 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802475 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802478 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802480 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802483 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802485 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802488 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802490 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802493 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802496 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802498 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802500 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802503 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802507 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802510 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802513 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:33:47.805332 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802515 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802517 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802521 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802524 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802526 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802529 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802532 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802534 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802536 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802539 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802542 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802544 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802547 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802549 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802551 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802554 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802556 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802559 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802561 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802564 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:33:47.805808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802934 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802940 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802943 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802946 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802949 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802952 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802955 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802958 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802960 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802963 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802966 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802968 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802971 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802974 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802976 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802979 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802981 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802984 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802987 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802989 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:33:47.806333 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802992 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802995 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.802997 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803000 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803002 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803005 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803017 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803021 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803024 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803027 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803030 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803032 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803035 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803037 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803040 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803043 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803046 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803048 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803050 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:33:47.806818 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803054 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803057 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803060 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803063 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803065 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803068 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803071 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803073 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803075 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803078 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803080 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803083 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803086 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803091 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803094 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803097 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803099 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803101 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803104 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:33:47.807336 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803106 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803109 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803112 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803114 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803116 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803119 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803121 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803123 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803126 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803128 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803132 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803135 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803137 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803139 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803144 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803148 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803151 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803153 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803156 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:33:47.807793 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803158 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803161 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803163 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803165 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803168 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803170 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803172 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803175 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803177 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803180 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803182 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803184 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803187 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803189 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803191 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803194 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803197 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803199 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803201 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803204 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:33:47.808284 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803207 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803210 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803287 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803302 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803315 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803319 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803323 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803328 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803333 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803337 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803341 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803344 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803347 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803350 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803353 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803356 2574 flags.go:64] FLAG: --cgroup-root="" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803359 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803362 2574 flags.go:64] FLAG: --client-ca-file="" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803365 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803367 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803375 2574 flags.go:64] FLAG: --cluster-domain="" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803378 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803382 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803385 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 17:33:47.808805 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803388 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803391 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803394 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803397 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803400 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803403 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803406 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803409 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803412 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803415 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803419 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803422 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803425 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803435 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803439 2574 flags.go:64] FLAG: --enable-server="true" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803444 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803452 2574 flags.go:64] FLAG: --event-burst="100" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803455 2574 flags.go:64] FLAG: --event-qps="50" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803458 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803461 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803464 2574 flags.go:64] FLAG: --eviction-hard="" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803467 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803470 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803473 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803476 2574 flags.go:64] FLAG: --eviction-soft="" Jul 08 17:33:47.809397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803479 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803482 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803484 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803487 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803491 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803493 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803496 2574 flags.go:64] FLAG: --feature-gates="" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803499 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803502 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803505 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803508 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803511 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803514 2574 flags.go:64] FLAG: --help="false" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803516 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-128-115.ec2.internal" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803519 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803522 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803524 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803528 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803531 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803534 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803536 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803545 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803549 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 17:33:47.810016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803555 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803558 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803561 2574 flags.go:64] FLAG: --kube-reserved="" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803564 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803566 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803569 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803572 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803575 2574 flags.go:64] FLAG: --lock-file="" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803578 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803580 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803583 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803588 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803591 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803594 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803596 2574 flags.go:64] FLAG: --logging-format="text" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803599 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803602 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803605 2574 flags.go:64] FLAG: --manifest-url="" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803608 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803614 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803617 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803621 2574 flags.go:64] FLAG: --max-pods="110" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803624 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803627 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803630 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 17:33:47.810663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803633 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803635 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803638 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803641 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803648 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803651 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803656 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803660 2574 flags.go:64] FLAG: --pod-cidr="" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803663 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803668 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803671 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803674 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803677 2574 flags.go:64] FLAG: --port="10250" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803680 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803683 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aa193013ae82ac48" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803686 2574 flags.go:64] FLAG: --qos-reserved="" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803689 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803691 2574 flags.go:64] FLAG: --register-node="true" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803694 2574 flags.go:64] FLAG: --register-with-taints="" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803698 2574 flags.go:64] FLAG: --registry-burst="10" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803700 2574 flags.go:64] FLAG: --registry-qps="5" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803703 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803706 2574 flags.go:64] FLAG: --reserved-memory="" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803709 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803712 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 17:33:47.811294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803715 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803718 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803721 2574 flags.go:64] FLAG: --runonce="false" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803723 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803726 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803729 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803732 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803734 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803737 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803740 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803743 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803746 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803748 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803751 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803757 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803760 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803763 2574 flags.go:64] FLAG: --system-cgroups="" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803766 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803771 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803774 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803777 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803781 2574 flags.go:64] FLAG: --tls-min-version="" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803784 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803786 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803789 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 17:33:47.811935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803792 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803795 2574 flags.go:64] FLAG: --v="2" Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803798 2574 flags.go:64] FLAG: --version="false" Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803802 2574 flags.go:64] FLAG: --vmodule="" Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803806 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.803809 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803961 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803966 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803968 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803971 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803973 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803976 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803978 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803980 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803983 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803985 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803988 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803990 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803992 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803995 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.803998 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:33:47.812579 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804003 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804012 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804015 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804017 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804019 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804022 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804024 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804027 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804030 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804032 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804035 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804037 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804039 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804042 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804044 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804046 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804049 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804051 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804054 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804056 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:33:47.813102 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804059 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804061 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804064 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804066 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804069 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804071 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804074 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804076 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804078 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804081 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804083 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804086 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804091 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804094 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804104 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804107 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804109 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804111 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804114 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804116 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:33:47.813591 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804119 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804121 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804125 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804129 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804132 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804135 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804138 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804140 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804143 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804145 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804148 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804150 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804152 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804155 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804157 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804160 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804163 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804165 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804169 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:33:47.814157 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804172 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804174 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804177 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804180 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804182 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804187 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804189 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804192 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804198 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804201 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804203 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804206 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804208 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804211 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804213 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804216 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804218 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804222 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804225 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804227 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:33:47.814633 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804230 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804232 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804235 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804237 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.804240 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.804894 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.811919 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.811941 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.811963 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.811968 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812031 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812036 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812039 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812041 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812044 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:33:47.815402 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812047 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812050 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812053 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812055 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812058 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812061 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812064 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812067 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812069 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812072 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812075 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812078 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812081 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812083 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812086 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812088 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812091 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812093 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812096 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:33:47.815786 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812098 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812101 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812103 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812107 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812111 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812114 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812118 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812120 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812123 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812126 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812128 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812131 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812133 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812136 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812138 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812140 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812143 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812147 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812150 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:33:47.816293 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812153 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812156 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812159 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812162 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812165 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812167 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812170 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812172 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812175 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812177 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812180 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812182 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812184 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812187 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812190 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812193 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812195 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812199 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812201 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812204 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:33:47.816805 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812206 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812209 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812211 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812214 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812216 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812218 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812221 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812223 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812225 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812228 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812230 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812232 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812235 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812238 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812240 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812243 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812246 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812248 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812250 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812253 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:33:47.817328 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812255 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812259 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812262 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812265 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812267 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812270 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812272 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812274 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812277 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812279 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812283 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812285 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812288 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812290 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812293 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:33:47.817808 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812295 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.812300 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812411 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812415 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812418 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812421 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812424 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812426 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812429 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812431 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812434 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812437 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812439 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812442 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:33:47.818213 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812445 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812448 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812451 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812453 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812456 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812458 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812461 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812464 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812466 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812469 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812471 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812474 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812476 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812479 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812482 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812484 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812486 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812489 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812491 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812494 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:33:47.818559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812496 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812498 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812501 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812503 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812505 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812508 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812510 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812513 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812515 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812517 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812520 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812522 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812525 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812527 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812530 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812533 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812536 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812539 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812542 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:33:47.819077 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812545 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812547 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812550 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812552 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812555 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812558 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812561 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812564 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812566 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812569 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812571 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812574 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812576 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812579 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812581 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812584 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812587 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812589 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812592 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812594 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812597 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:33:47.819530 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812599 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812602 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812604 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812607 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812609 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812611 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812614 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812616 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812619 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812621 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812624 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812626 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812628 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812631 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812633 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812636 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812638 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812640 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812643 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812645 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:33:47.820197 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812648 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812650 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812654 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812657 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812660 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812662 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:47.812665 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.812669 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.812783 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.816525 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.818004 2574 server.go:1019] "Starting client certificate rotation" Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.818106 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.818138 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:33:47.820708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.820645 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:33:47.822735 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.822722 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:33:47.835321 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.835302 2574 log.go:25] "Validated CRI v1 runtime API" Jul 08 17:33:47.837665 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.837649 2574 log.go:25] "Validated CRI v1 image API" Jul 08 17:33:47.839202 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.839188 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 17:33:47.845268 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.845247 2574 fs.go:135] Filesystem UUIDs: map[42eec73f-98f0-4ff3-9e3a-5f534c002e19:/dev/nvme0n1p4 47825b7f-825d-490e-a006-f585184218f9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 08 17:33:47.845321 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.845268 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 17:33:47.845547 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.845526 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:33:47.851423 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.851314 2574 manager.go:216] Machine: {Timestamp:2026-07-08 17:33:47.850012127 +0000 UTC m=+0.349791861 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107685 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2157382529172fb4f1f7ff0bc4c749 SystemUUID:ec215738-2529-172f-b4f1-f7ff0bc4c749 BootID:3e8e4756-6376-40fa-8321-a505c92c589d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d8:c8:8c:67:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d8:c8:8c:67:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:9c:fb:ac:a6:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 17:33:47.851423 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.851421 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 17:33:47.851528 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.851502 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 17:33:47.854553 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.854523 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 17:33:47.854696 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.854556 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-115.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 17:33:47.855329 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.855319 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 17:33:47.855367 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.855331 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 17:33:47.855367 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.855338 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:33:47.855951 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.855941 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:33:47.857792 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.857783 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 17:33:47.861641 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861628 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:33:47.861766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861757 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 17:33:47.861803 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861789 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 08 17:33:47.861846 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861803 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 17:33:47.861846 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861838 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 17:33:47.861903 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861851 2574 kubelet.go:398] "Adding apiserver pod source" Jul 08 17:33:47.861903 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.861863 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 17:33:47.862296 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.862283 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:33:47.862340 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.862302 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:33:47.865447 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.865214 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 17:33:47.868275 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.868261 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 17:33:47.868328 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.868292 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 17:33:47.869598 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869586 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869604 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869610 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869615 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869624 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869629 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869637 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 17:33:47.869645 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869643 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 17:33:47.869886 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869649 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 17:33:47.869886 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869655 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 17:33:47.869886 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869664 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 17:33:47.869886 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.869670 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 17:33:47.870501 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.870488 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 17:33:47.870501 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.870501 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 17:33:47.870706 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.870680 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 17:33:47.870753 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.870680 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 17:33:47.874081 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.874066 2574 server.go:1268] "Started kubelet" Jul 08 17:33:47.874155 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.874118 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 17:33:47.874226 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.874199 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 17:33:47.874258 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.874246 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 17:33:47.874761 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.874737 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-115.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 17:33:47.875132 ip-10-0-128-115 systemd[1]: Started Kubernetes Kubelet. Jul 08 17:33:47.876021 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.875960 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 17:33:47.876874 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.876855 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 08 17:33:47.879950 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.879136 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-115.ec2.internal.18c060ee06f6baa0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-115.ec2.internal,UID:ip-10-0-128-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-115.ec2.internal,},FirstTimestamp:2026-07-08 17:33:47.874032288 +0000 UTC m=+0.373812055,LastTimestamp:2026-07-08 17:33:47.874032288 +0000 UTC m=+0.373812055,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-115.ec2.internal,}" Jul 08 17:33:47.880534 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.880504 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 17:33:47.883976 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.883961 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 17:33:47.883976 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.883971 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 17:33:47.884096 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.884085 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:47.884780 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.884758 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 17:33:47.884780 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.884769 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 17:33:47.884780 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.884778 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 17:33:47.884974 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.884872 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 08 17:33:47.884974 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.884881 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 08 17:33:47.886139 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.886107 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 17:33:47.886234 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.886167 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 17:33:47.886418 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886400 2574 factory.go:55] Registering systemd factory Jul 08 17:33:47.886418 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886419 2574 factory.go:223] Registration of the systemd container factory successfully Jul 08 17:33:47.886867 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886850 2574 factory.go:153] Registering CRI-O factory Jul 08 17:33:47.886867 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886869 2574 factory.go:223] Registration of the crio container factory successfully Jul 08 17:33:47.887035 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886927 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 17:33:47.887035 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886951 2574 factory.go:103] Registering Raw factory Jul 08 17:33:47.887035 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.886968 2574 manager.go:1196] Started watching for new ooms in manager Jul 08 17:33:47.887517 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.887505 2574 manager.go:349] Starting recovery of all containers Jul 08 17:33:47.892538 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.892493 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 17:33:47.899663 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.899476 2574 manager.go:354] Recovery completed Jul 08 17:33:47.900427 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.900356 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjczd" Jul 08 17:33:47.904638 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.904625 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:47.907440 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.907421 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:47.907522 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.907451 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:47.907522 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.907465 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:47.908118 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908097 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 17:33:47.908118 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908110 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 17:33:47.908118 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908113 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:47.908266 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908130 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:33:47.908671 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908655 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjczd" Jul 08 17:33:47.908839 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908811 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:47.908896 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908859 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:47.908896 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.908869 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:47.910277 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.910265 2574 policy_none.go:49] "None policy: Start" Jul 08 17:33:47.910342 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.910284 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 17:33:47.910342 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.910298 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 17:33:47.911610 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.911597 2574 policy_none.go:47] "Start" Jul 08 17:33:47.948954 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.948938 2574 manager.go:337] "Starting Device Plugin manager" Jul 08 17:33:47.949041 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.948978 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 17:33:47.949041 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.948988 2574 server.go:87] "Starting device plugin registration server" Jul 08 17:33:47.949266 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.949252 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 17:33:47.949334 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.949270 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 17:33:47.949384 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.949353 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 17:33:47.949437 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.949429 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 17:33:47.949482 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:47.949437 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 17:33:47.950007 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.949990 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 17:33:47.950109 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:47.950031 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.050387 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.050304 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:48.051607 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.051466 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:48.051607 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.051504 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:48.051607 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.051514 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:48.051607 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.051542 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.057503 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.057483 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 17:33:48.057589 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.057524 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 17:33:48.057589 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.057543 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 17:33:48.057589 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.057581 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 17:33:48.058137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.058120 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.058224 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.058147 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-115.ec2.internal\": node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.059748 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.059733 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:33:48.103947 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.103911 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.158059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.158021 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal"] Jul 08 17:33:48.158145 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.158133 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:48.159110 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.159093 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:48.159178 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.159126 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:48.159178 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.159136 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:48.160611 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.160599 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:48.160729 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.160714 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.160765 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.160743 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:48.161328 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.161315 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:48.161396 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.161316 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:48.161396 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.161368 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:48.161396 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.161381 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:48.161504 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.161347 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:48.161504 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.161425 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:48.163017 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.163001 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.163077 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.163040 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:33:48.163807 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.163788 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:33:48.163906 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.163837 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:33:48.163906 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.163857 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:33:48.185684 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.185659 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-115.ec2.internal\" not found" node="ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.190091 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.190071 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-115.ec2.internal\" not found" node="ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.204644 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.204624 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.286000 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.285963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/385ef6a0a0ac9ab0104bcd0e7cc3e5b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal\" (UID: \"385ef6a0a0ac9ab0104bcd0e7cc3e5b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.286000 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.286000 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/385ef6a0a0ac9ab0104bcd0e7cc3e5b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal\" (UID: \"385ef6a0a0ac9ab0104bcd0e7cc3e5b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.286168 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.286081 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/57192b8ecd12bde2baff3a89b0094a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-115.ec2.internal\" (UID: \"57192b8ecd12bde2baff3a89b0094a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.305100 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.305042 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.386802 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.386763 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/385ef6a0a0ac9ab0104bcd0e7cc3e5b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal\" (UID: \"385ef6a0a0ac9ab0104bcd0e7cc3e5b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.386802 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.386807 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/385ef6a0a0ac9ab0104bcd0e7cc3e5b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal\" (UID: \"385ef6a0a0ac9ab0104bcd0e7cc3e5b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.386964 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.386841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/57192b8ecd12bde2baff3a89b0094a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-115.ec2.internal\" (UID: \"57192b8ecd12bde2baff3a89b0094a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.386964 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.386883 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/385ef6a0a0ac9ab0104bcd0e7cc3e5b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal\" (UID: \"385ef6a0a0ac9ab0104bcd0e7cc3e5b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.386964 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.386889 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/385ef6a0a0ac9ab0104bcd0e7cc3e5b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal\" (UID: \"385ef6a0a0ac9ab0104bcd0e7cc3e5b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.386964 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.386938 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/57192b8ecd12bde2baff3a89b0094a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-115.ec2.internal\" (UID: \"57192b8ecd12bde2baff3a89b0094a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.405908 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.405878 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.489398 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.489362 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.494157 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.493612 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.506320 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.506298 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.607749 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.607699 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-115.ec2.internal\" not found" Jul 08 17:33:48.692858 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.692824 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:33:48.785718 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.785675 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.799559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.799534 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:33:48.801453 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.801437 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" Jul 08 17:33:48.812167 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.812104 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:33:48.817222 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.817202 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 17:33:48.817342 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.817324 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:33:48.817399 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.817377 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:33:48.863352 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.863312 2574 apiserver.go:52] "Watching apiserver" Jul 08 17:33:48.878995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.878968 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 17:33:48.879350 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.879325 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-vt95p","kube-system/global-pull-secret-syncer-sgg9w","kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal","openshift-cluster-node-tuning-operator/tuned-75qzd","openshift-multus/multus-additional-cni-plugins-7cfqv","openshift-multus/multus-nh469","openshift-multus/network-metrics-daemon-jwzwj","openshift-network-operator/iptables-alerter-lq2sd","kube-system/konnectivity-agent-rk9qs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk","openshift-dns/node-resolver-zv4d4","openshift-image-registry/node-ca-s5tj7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal","openshift-network-diagnostics/network-check-target-sf5mq"] Jul 08 17:33:48.881094 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.881058 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:48.881199 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.881137 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:33:48.882370 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.882351 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.885415 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.883991 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:48.885415 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.884077 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:33:48.885415 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.884429 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 17:33:48.885415 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.884690 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.885415 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.884948 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2svwm\"" Jul 08 17:33:48.885727 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.885406 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.885809 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.885787 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.887922 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.887902 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.888130 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.888112 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.888462 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.888447 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.888673 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.888659 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nmqqq\"" Jul 08 17:33:48.889341 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.889327 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nh469" Jul 08 17:33:48.890232 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890213 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.890459 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890441 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 17:33:48.890538 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890444 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 17:33:48.890599 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890580 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:48.890669 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890652 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.890722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890660 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysconfig\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.890834 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-systemd\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.890891 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890867 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-sys\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.890978 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-var-lib-kubelet\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.890978 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbj5s\" (UniqueName: \"kubernetes.io/projected/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-kube-api-access-gbj5s\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:48.890978 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.890958 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b713581-56c8-46be-bc7c-51279b58448c-tmp-dir\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.891137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-kubernetes\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysctl-d\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891058 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-run\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-host\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891090 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7m8tw\"" Jul 08 17:33:48.891137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891090 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891128 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac6903ba-e074-434e-bb41-2f93a0c29d63-tmp\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891194 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891220 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b713581-56c8-46be-bc7c-51279b58448c-hosts-file\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891262 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysctl-conf\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-lib-modules\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891372 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-tuned\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44cfw\" (UniqueName: \"kubernetes.io/projected/ac6903ba-e074-434e-bb41-2f93a0c29d63-kube-api-access-44cfw\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891403 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 17:33:48.891866 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891433 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj49w\" (UniqueName: \"kubernetes.io/projected/3b713581-56c8-46be-bc7c-51279b58448c-kube-api-access-mj49w\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.891866 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891465 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-modprobe-d\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.891866 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.891675 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v2gbg\"" Jul 08 17:33:48.892607 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.892588 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:48.892999 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.892978 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 17:33:48.893096 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.893035 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8rs65\"" Jul 08 17:33:48.893512 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.893496 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 17:33:48.893956 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.893939 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:48.894035 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.894000 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:33:48.894924 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.894906 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.895019 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.894946 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.895239 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.895216 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 17:33:48.895333 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.895228 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jwrcm\"" Jul 08 17:33:48.895400 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.895350 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.896869 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.896851 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.897391 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.897372 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zkskc\"" Jul 08 17:33:48.897473 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.897377 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.897624 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.897606 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 17:33:48.897695 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.897645 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.898356 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.898337 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:48.899276 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899255 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:33:48.899355 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899282 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.899706 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899687 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 17:33:48.899706 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899693 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 17:33:48.899814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899709 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fdql8\"" Jul 08 17:33:48.899917 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899882 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 17:33:48.899985 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.899919 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 17:33:48.900027 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.900009 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.900369 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.900352 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 17:33:48.901343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.901327 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 17:33:48.901343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.901337 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5c4cn\"" Jul 08 17:33:48.901470 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.901378 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 17:33:48.913358 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.913334 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 17:28:48 +0000 UTC" deadline="2028-02-16 13:12:01.170484282 +0000 UTC" Jul 08 17:33:48.913358 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.913356 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14107h38m12.257130151s" Jul 08 17:33:48.930187 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.930160 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-drfs2" Jul 08 17:33:48.940747 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.940727 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-drfs2" Jul 08 17:33:48.986201 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.986182 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 17:33:48.992330 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992311 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7xw9\" (UniqueName: \"kubernetes.io/projected/744b8ce9-1887-478f-a460-b648e27cc580-kube-api-access-w7xw9\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:48.992390 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992337 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-cnibin\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.992390 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992361 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-hostroot\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.992390 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992380 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-var-lib-kubelet\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.992524 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992396 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-slash\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.992524 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992410 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-cni-bin\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.992524 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992436 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-ovnkube-config\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.992524 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992480 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-var-lib-kubelet\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.992524 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992498 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/064068a9-6858-474d-a457-b2bc3fb6c435-multus-daemon-config\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992540 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eae2679f-b62d-406a-bf32-f5130af69481-iptables-alerter-script\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992569 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42a39394-26a1-4f87-8f51-709a58922414-kubelet-config\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992586 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-socket-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysctl-d\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-host\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-kubelet\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992672 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992688 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42a39394-26a1-4f87-8f51-709a58922414-dbus\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992702 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-host\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992726 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr757\" (UniqueName: \"kubernetes.io/projected/f79f10bb-b4af-43ee-a238-3c9e8949b116-kube-api-access-rr757\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.992794 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992765 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-tuned\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.992857 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992868 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysctl-d\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992874 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af2773a3-d6ec-4fd1-8bf1-81440f6c11cb-agent-certs\") pod \"konnectivity-agent-rk9qs\" (UID: \"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb\") " pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:48.992947 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:33:49.492914078 +0000 UTC m=+1.992693797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.992971 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-etc-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993005 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-node-log\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993059 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-log-socket\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-os-release\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993123 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-os-release\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993134 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-systemd\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993178 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-sys\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993202 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-var-lib-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993217 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-systemd\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-k8s-cni-cncf-io\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.993354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-sys-fs\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993262 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-sys\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac6903ba-e074-434e-bb41-2f93a0c29d63-tmp\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993312 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-run-ovn-kubernetes\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-system-cni-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993406 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993454 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysctl-conf\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993499 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3655e63d-2e5a-408d-8af6-b1b312194629-ovn-node-metrics-cert\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993537 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-conf-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993557 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-multus-certs\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993578 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-registration-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44cfw\" (UniqueName: \"kubernetes.io/projected/ac6903ba-e074-434e-bb41-2f93a0c29d63-kube-api-access-44cfw\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysctl-conf\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993690 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-run-netns\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993776 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-cni-netd\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/744b8ce9-1887-478f-a460-b648e27cc580-host\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993857 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj49w\" (UniqueName: \"kubernetes.io/projected/3b713581-56c8-46be-bc7c-51279b58448c-kube-api-access-mj49w\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993912 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-modprobe-d\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-system-cni-dir\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-etc-kubernetes\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.993978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-systemd-units\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-kubernetes\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994040 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-modprobe-d\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994066 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-kubernetes\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994123 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-ovn\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994257 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cnibin\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk5mj\" (UniqueName: \"kubernetes.io/projected/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-kube-api-access-qk5mj\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994304 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-cni-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-etc-selinux\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.994737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994344 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af2773a3-d6ec-4fd1-8bf1-81440f6c11cb-konnectivity-ca\") pod \"konnectivity-agent-rk9qs\" (UID: \"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb\") " pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-systemd\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-ovnkube-script-lib\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994441 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8qxd\" (UniqueName: \"kubernetes.io/projected/eae2679f-b62d-406a-bf32-f5130af69481-kube-api-access-j8qxd\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-device-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994492 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp5bw\" (UniqueName: \"kubernetes.io/projected/3655e63d-2e5a-408d-8af6-b1b312194629-kube-api-access-sp5bw\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/744b8ce9-1887-478f-a460-b648e27cc580-serviceca\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994580 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l78kk\" (UniqueName: \"kubernetes.io/projected/064068a9-6858-474d-a457-b2bc3fb6c435-kube-api-access-l78kk\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysconfig\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994733 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/064068a9-6858-474d-a457-b2bc3fb6c435-cni-binary-copy\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-socket-dir-parent\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994764 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-sysconfig\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-netns\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994861 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-cni-bin\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994892 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-cni-multus\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbj5s\" (UniqueName: \"kubernetes.io/projected/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-kube-api-access-gbj5s\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:48.995559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.994957 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b713581-56c8-46be-bc7c-51279b58448c-tmp-dir\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-run\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995025 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eae2679f-b62d-406a-bf32-f5130af69481-host-slash\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995052 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b713581-56c8-46be-bc7c-51279b58448c-hosts-file\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-lib-modules\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995101 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-env-overrides\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995116 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-run\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995128 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-kubelet\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995251 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac6903ba-e074-434e-bb41-2f93a0c29d63-lib-modules\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995494 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b713581-56c8-46be-bc7c-51279b58448c-tmp-dir\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.996249 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.995559 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b713581-56c8-46be-bc7c-51279b58448c-hosts-file\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:48.996690 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.996673 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac6903ba-e074-434e-bb41-2f93a0c29d63-etc-tuned\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:48.996748 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:48.996727 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac6903ba-e074-434e-bb41-2f93a0c29d63-tmp\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:49.008187 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.008151 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:33:49.008187 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.008179 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:33:49.008187 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.008193 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:49.008417 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.008270 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:49.508249906 +0000 UTC m=+2.008029630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:49.009585 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.009556 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj49w\" (UniqueName: \"kubernetes.io/projected/3b713581-56c8-46be-bc7c-51279b58448c-kube-api-access-mj49w\") pod \"node-resolver-zv4d4\" (UID: \"3b713581-56c8-46be-bc7c-51279b58448c\") " pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:49.009659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.009649 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbj5s\" (UniqueName: \"kubernetes.io/projected/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-kube-api-access-gbj5s\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:49.010391 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.010375 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44cfw\" (UniqueName: \"kubernetes.io/projected/ac6903ba-e074-434e-bb41-2f93a0c29d63-kube-api-access-44cfw\") pod \"tuned-75qzd\" (UID: \"ac6903ba-e074-434e-bb41-2f93a0c29d63\") " pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:49.019026 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.018997 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:33:49.096477 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-system-cni-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.096477 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:49.096477 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096445 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3655e63d-2e5a-408d-8af6-b1b312194629-ovn-node-metrics-cert\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.096477 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096551 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-system-cni-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.096570 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096611 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-conf-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096698 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-conf-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096709 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-multus-certs\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096751 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-multus-certs\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.096787 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.096758 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:49.596735085 +0000 UTC m=+2.096514808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-registration-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096865 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-run-netns\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-cni-netd\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/744b8ce9-1887-478f-a460-b648e27cc580-host\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096957 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-registration-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096973 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-system-cni-dir\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-run-netns\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.096989 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-cni-netd\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097015 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/744b8ce9-1887-478f-a460-b648e27cc580-host\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097022 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-system-cni-dir\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097052 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-etc-kubernetes\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097087 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-etc-kubernetes\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097088 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-systemd-units\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097120 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-systemd-units\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097129 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-ovn\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097154 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097159 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cnibin\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097173 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cnibin\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk5mj\" (UniqueName: \"kubernetes.io/projected/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-kube-api-access-qk5mj\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097217 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-ovn\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097230 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-cni-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097260 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-etc-selinux\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af2773a3-d6ec-4fd1-8bf1-81440f6c11cb-konnectivity-ca\") pod \"konnectivity-agent-rk9qs\" (UID: \"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb\") " pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097327 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-cni-dir\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-systemd\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-ovnkube-script-lib\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097371 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8qxd\" (UniqueName: \"kubernetes.io/projected/eae2679f-b62d-406a-bf32-f5130af69481-kube-api-access-j8qxd\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097398 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-device-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097424 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-systemd\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.097804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097427 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp5bw\" (UniqueName: \"kubernetes.io/projected/3655e63d-2e5a-408d-8af6-b1b312194629-kube-api-access-sp5bw\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097472 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/744b8ce9-1887-478f-a460-b648e27cc580-serviceca\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097497 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-device-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097541 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-etc-selinux\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097513 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l78kk\" (UniqueName: \"kubernetes.io/projected/064068a9-6858-474d-a457-b2bc3fb6c435-kube-api-access-l78kk\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097777 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/064068a9-6858-474d-a457-b2bc3fb6c435-cni-binary-copy\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-socket-dir-parent\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af2773a3-d6ec-4fd1-8bf1-81440f6c11cb-konnectivity-ca\") pod \"konnectivity-agent-rk9qs\" (UID: \"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb\") " pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097936 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-multus-socket-dir-parent\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097964 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/744b8ce9-1887-478f-a460-b648e27cc580-serviceca\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097906 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.097972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-netns\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-netns\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098043 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-cni-bin\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-cni-multus\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098078 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-cni-bin\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.098659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098102 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eae2679f-b62d-406a-bf32-f5130af69481-host-slash\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098126 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-cni-multus\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098133 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-env-overrides\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098155 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eae2679f-b62d-406a-bf32-f5130af69481-host-slash\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098166 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-kubelet\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7xw9\" (UniqueName: \"kubernetes.io/projected/744b8ce9-1887-478f-a460-b648e27cc580-kube-api-access-w7xw9\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-cnibin\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098266 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-var-lib-kubelet\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-hostroot\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098339 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-hostroot\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098380 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-cnibin\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098443 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-slash\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098455 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-ovnkube-script-lib\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098478 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/064068a9-6858-474d-a457-b2bc3fb6c435-cni-binary-copy\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-cni-bin\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098514 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-slash\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-ovnkube-config\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098540 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-cni-bin\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.099531 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/064068a9-6858-474d-a457-b2bc3fb6c435-multus-daemon-config\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098578 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-env-overrides\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098587 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eae2679f-b62d-406a-bf32-f5130af69481-iptables-alerter-script\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098642 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42a39394-26a1-4f87-8f51-709a58922414-kubelet-config\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098680 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-socket-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-kubelet\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098732 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098751 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/42a39394-26a1-4f87-8f51-709a58922414-kubelet-config\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098797 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42a39394-26a1-4f87-8f51-709a58922414-dbus\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-kubelet\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098842 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr757\" (UniqueName: \"kubernetes.io/projected/f79f10bb-b4af-43ee-a238-3c9e8949b116-kube-api-access-rr757\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098934 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098941 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-socket-dir\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.098972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af2773a3-d6ec-4fd1-8bf1-81440f6c11cb-agent-certs\") pod \"konnectivity-agent-rk9qs\" (UID: \"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb\") " pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099002 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-etc-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099014 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/42a39394-26a1-4f87-8f51-709a58922414-dbus\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:49.100175 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099028 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099054 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-node-log\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099061 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3655e63d-2e5a-408d-8af6-b1b312194629-ovnkube-config\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-log-socket\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099121 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-run-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099122 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-log-socket\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099152 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-node-log\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099173 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-etc-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099179 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/064068a9-6858-474d-a457-b2bc3fb6c435-multus-daemon-config\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-os-release\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-os-release\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099237 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-var-lib-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099259 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-os-release\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-k8s-cni-cncf-io\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099268 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eae2679f-b62d-406a-bf32-f5130af69481-iptables-alerter-script\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099292 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-host-run-k8s-cni-cncf-io\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099301 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-sys-fs\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099312 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/064068a9-6858-474d-a457-b2bc3fb6c435-os-release\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.100722 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099341 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-var-lib-openvswitch\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-run-ovn-kubernetes\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099377 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f79f10bb-b4af-43ee-a238-3c9e8949b116-sys-fs\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099417 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3655e63d-2e5a-408d-8af6-b1b312194629-host-run-ovn-kubernetes\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099588 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3655e63d-2e5a-408d-8af6-b1b312194629-ovn-node-metrics-cert\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099635 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-cni-binary-copy\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.101263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.099839 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.101891 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.101862 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af2773a3-d6ec-4fd1-8bf1-81440f6c11cb-agent-certs\") pod \"konnectivity-agent-rk9qs\" (UID: \"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb\") " pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:49.111219 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.111194 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk5mj\" (UniqueName: \"kubernetes.io/projected/0ecc90ad-bc13-4f07-9e16-a25dd51d9331-kube-api-access-qk5mj\") pod \"multus-additional-cni-plugins-7cfqv\" (UID: \"0ecc90ad-bc13-4f07-9e16-a25dd51d9331\") " pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.115324 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.115282 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr757\" (UniqueName: \"kubernetes.io/projected/f79f10bb-b4af-43ee-a238-3c9e8949b116-kube-api-access-rr757\") pod \"aws-ebs-csi-driver-node-v7fjk\" (UID: \"f79f10bb-b4af-43ee-a238-3c9e8949b116\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.115536 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.115453 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l78kk\" (UniqueName: \"kubernetes.io/projected/064068a9-6858-474d-a457-b2bc3fb6c435-kube-api-access-l78kk\") pod \"multus-nh469\" (UID: \"064068a9-6858-474d-a457-b2bc3fb6c435\") " pod="openshift-multus/multus-nh469" Jul 08 17:33:49.115650 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.115603 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp5bw\" (UniqueName: \"kubernetes.io/projected/3655e63d-2e5a-408d-8af6-b1b312194629-kube-api-access-sp5bw\") pod \"ovnkube-node-vt95p\" (UID: \"3655e63d-2e5a-408d-8af6-b1b312194629\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.116140 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.116112 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7xw9\" (UniqueName: \"kubernetes.io/projected/744b8ce9-1887-478f-a460-b648e27cc580-kube-api-access-w7xw9\") pod \"node-ca-s5tj7\" (UID: \"744b8ce9-1887-478f-a460-b648e27cc580\") " pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.116363 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.116341 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8qxd\" (UniqueName: \"kubernetes.io/projected/eae2679f-b62d-406a-bf32-f5130af69481-kube-api-access-j8qxd\") pod \"iptables-alerter-lq2sd\" (UID: \"eae2679f-b62d-406a-bf32-f5130af69481\") " pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.201573 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.201535 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zv4d4" Jul 08 17:33:49.225489 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.225458 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-75qzd" Jul 08 17:33:49.234442 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.234416 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" Jul 08 17:33:49.246516 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.246491 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nh469" Jul 08 17:33:49.257552 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.257507 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6903ba_e074_434e_bb41_2f93a0c29d63.slice/crio-81d1bc221d595178e3df3527a16fc0d6dcdafb1b9aff6314ae2e84ed6c16bbfe WatchSource:0}: Error finding container 81d1bc221d595178e3df3527a16fc0d6dcdafb1b9aff6314ae2e84ed6c16bbfe: Status 404 returned error can't find the container with id 81d1bc221d595178e3df3527a16fc0d6dcdafb1b9aff6314ae2e84ed6c16bbfe Jul 08 17:33:49.257992 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.257966 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57192b8ecd12bde2baff3a89b0094a6a.slice/crio-b4d3481fa6df878224291eadf0ea32133d6bb8c7fab3bd5aa32a4dfb2ed00b15 WatchSource:0}: Error finding container b4d3481fa6df878224291eadf0ea32133d6bb8c7fab3bd5aa32a4dfb2ed00b15: Status 404 returned error can't find the container with id b4d3481fa6df878224291eadf0ea32133d6bb8c7fab3bd5aa32a4dfb2ed00b15 Jul 08 17:33:49.258249 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.258221 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod385ef6a0a0ac9ab0104bcd0e7cc3e5b2.slice/crio-37ebfc83156dc466ae3cb5d1440412c40c480fc6385a86f3a5fab5ea056ad45b WatchSource:0}: Error finding container 37ebfc83156dc466ae3cb5d1440412c40c480fc6385a86f3a5fab5ea056ad45b: Status 404 returned error can't find the container with id 37ebfc83156dc466ae3cb5d1440412c40c480fc6385a86f3a5fab5ea056ad45b Jul 08 17:33:49.258658 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.258638 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod064068a9_6858_474d_a457_b2bc3fb6c435.slice/crio-4cb3af84a9ae08d18f992975d806dbb938c9b9a80ffce21b6fd95d1071cab05b WatchSource:0}: Error finding container 4cb3af84a9ae08d18f992975d806dbb938c9b9a80ffce21b6fd95d1071cab05b: Status 404 returned error can't find the container with id 4cb3af84a9ae08d18f992975d806dbb938c9b9a80ffce21b6fd95d1071cab05b Jul 08 17:33:49.259476 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.259459 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecc90ad_bc13_4f07_9e16_a25dd51d9331.slice/crio-7cad7721051c12b23d8352a9e4303a8717b9fc9c705a3d1f49d562276d3cdc87 WatchSource:0}: Error finding container 7cad7721051c12b23d8352a9e4303a8717b9fc9c705a3d1f49d562276d3cdc87: Status 404 returned error can't find the container with id 7cad7721051c12b23d8352a9e4303a8717b9fc9c705a3d1f49d562276d3cdc87 Jul 08 17:33:49.263603 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.263478 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:33:49.265969 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.265946 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:33:49.275523 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.275500 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf2773a3_d6ec_4fd1_8bf1_81440f6c11cb.slice/crio-bfd404db7d19fb6ffaf6c8cbee848a5727254819280f67e49492f4ad3be91d7c WatchSource:0}: Error finding container bfd404db7d19fb6ffaf6c8cbee848a5727254819280f67e49492f4ad3be91d7c: Status 404 returned error can't find the container with id bfd404db7d19fb6ffaf6c8cbee848a5727254819280f67e49492f4ad3be91d7c Jul 08 17:33:49.283629 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.283607 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lq2sd" Jul 08 17:33:49.291559 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.291536 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae2679f_b62d_406a_bf32_f5130af69481.slice/crio-4137280fdaece1b222795ff4bff4f25447674097cf54414a6149b2df0c035807 WatchSource:0}: Error finding container 4137280fdaece1b222795ff4bff4f25447674097cf54414a6149b2df0c035807: Status 404 returned error can't find the container with id 4137280fdaece1b222795ff4bff4f25447674097cf54414a6149b2df0c035807 Jul 08 17:33:49.297508 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.297493 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" Jul 08 17:33:49.305661 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.305639 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79f10bb_b4af_43ee_a238_3c9e8949b116.slice/crio-7c2571a7d781d39d44dedefc0898a8487a4aea99564f4a83235a5fd083e5590d WatchSource:0}: Error finding container 7c2571a7d781d39d44dedefc0898a8487a4aea99564f4a83235a5fd083e5590d: Status 404 returned error can't find the container with id 7c2571a7d781d39d44dedefc0898a8487a4aea99564f4a83235a5fd083e5590d Jul 08 17:33:49.321286 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.321266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:33:49.326569 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.326549 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s5tj7" Jul 08 17:33:49.331219 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.330242 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3655e63d_2e5a_408d_8af6_b1b312194629.slice/crio-1466868e49951aeace01bddf26e9b17b0a30867b0a94a811e29afaa85953321a WatchSource:0}: Error finding container 1466868e49951aeace01bddf26e9b17b0a30867b0a94a811e29afaa85953321a: Status 404 returned error can't find the container with id 1466868e49951aeace01bddf26e9b17b0a30867b0a94a811e29afaa85953321a Jul 08 17:33:49.336800 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:33:49.336780 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744b8ce9_1887_478f_a460_b648e27cc580.slice/crio-b099c5389cc0910cc68c2e5011ae18fcf5d323f242e415bad03e2f446e818d7d WatchSource:0}: Error finding container b099c5389cc0910cc68c2e5011ae18fcf5d323f242e415bad03e2f446e818d7d: Status 404 returned error can't find the container with id b099c5389cc0910cc68c2e5011ae18fcf5d323f242e415bad03e2f446e818d7d Jul 08 17:33:49.368084 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.367981 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:33:49.502846 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.502797 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:49.503019 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.502938 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:49.503019 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.502998 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:33:50.502983202 +0000 UTC m=+3.002762905 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:49.604008 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.603965 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:49.604008 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.604005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:49.604282 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.604139 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:49.604282 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.604200 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:33:49.604282 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.604211 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:50.604194787 +0000 UTC m=+3.103974492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:49.604282 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.604221 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:33:49.604282 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.604232 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:49.604559 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:49.604288 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:50.604268732 +0000 UTC m=+3.104048440 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:49.656215 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.656132 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:33:49.942814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.942696 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:28:49 +0000 UTC" deadline="2028-04-05 08:34:22.513290188 +0000 UTC" Jul 08 17:33:49.942814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:49.942765 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15279h0m32.570529257s" Jul 08 17:33:50.059198 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.059158 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:50.059382 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.059323 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:33:50.060922 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.060875 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:50.061052 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.061017 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:33:50.072373 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.072319 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zv4d4" event={"ID":"3b713581-56c8-46be-bc7c-51279b58448c","Type":"ContainerStarted","Data":"d2424eaf70164468ab169761cae2b1b2febbe6d52186c7aa09d47b237bd831b7"} Jul 08 17:33:50.074460 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.074372 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s5tj7" event={"ID":"744b8ce9-1887-478f-a460-b648e27cc580","Type":"ContainerStarted","Data":"b099c5389cc0910cc68c2e5011ae18fcf5d323f242e415bad03e2f446e818d7d"} Jul 08 17:33:50.094937 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.091595 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lq2sd" event={"ID":"eae2679f-b62d-406a-bf32-f5130af69481","Type":"ContainerStarted","Data":"4137280fdaece1b222795ff4bff4f25447674097cf54414a6149b2df0c035807"} Jul 08 17:33:50.107057 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.106977 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rk9qs" event={"ID":"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb","Type":"ContainerStarted","Data":"bfd404db7d19fb6ffaf6c8cbee848a5727254819280f67e49492f4ad3be91d7c"} Jul 08 17:33:50.132571 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.132534 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nh469" event={"ID":"064068a9-6858-474d-a457-b2bc3fb6c435","Type":"ContainerStarted","Data":"4cb3af84a9ae08d18f992975d806dbb938c9b9a80ffce21b6fd95d1071cab05b"} Jul 08 17:33:50.149224 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.149151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerStarted","Data":"7cad7721051c12b23d8352a9e4303a8717b9fc9c705a3d1f49d562276d3cdc87"} Jul 08 17:33:50.167817 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.167773 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"1466868e49951aeace01bddf26e9b17b0a30867b0a94a811e29afaa85953321a"} Jul 08 17:33:50.172387 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.172354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" event={"ID":"f79f10bb-b4af-43ee-a238-3c9e8949b116","Type":"ContainerStarted","Data":"7c2571a7d781d39d44dedefc0898a8487a4aea99564f4a83235a5fd083e5590d"} Jul 08 17:33:50.176267 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.176234 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" event={"ID":"57192b8ecd12bde2baff3a89b0094a6a","Type":"ContainerStarted","Data":"b4d3481fa6df878224291eadf0ea32133d6bb8c7fab3bd5aa32a4dfb2ed00b15"} Jul 08 17:33:50.186617 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.186592 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" event={"ID":"385ef6a0a0ac9ab0104bcd0e7cc3e5b2","Type":"ContainerStarted","Data":"37ebfc83156dc466ae3cb5d1440412c40c480fc6385a86f3a5fab5ea056ad45b"} Jul 08 17:33:50.188538 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.188512 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-75qzd" event={"ID":"ac6903ba-e074-434e-bb41-2f93a0c29d63","Type":"ContainerStarted","Data":"81d1bc221d595178e3df3527a16fc0d6dcdafb1b9aff6314ae2e84ed6c16bbfe"} Jul 08 17:33:50.427471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.427382 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:33:50.512560 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.512520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:50.512744 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.512726 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:50.512830 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.512808 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:33:52.512785289 +0000 UTC m=+5.012565010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.612968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.613022 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.613211 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.613230 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.613245 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.613309 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:52.613288266 +0000 UTC m=+5.113067973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.613735 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:50.614639 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:50.613787 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:52.613771805 +0000 UTC m=+5.113551532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:50.943662 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.943587 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:28:49 +0000 UTC" deadline="2028-04-02 18:33:19.214371926 +0000 UTC" Jul 08 17:33:50.943662 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:50.943628 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15216h59m28.27074927s" Jul 08 17:33:51.058162 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:51.058071 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:51.058345 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:51.058210 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:33:52.061315 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:52.060760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:52.061315 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.060924 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:33:52.061315 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:52.061040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:52.061315 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.061170 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:33:52.534615 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:52.534424 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:52.534798 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.534627 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:52.534798 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.534710 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:33:56.534688403 +0000 UTC m=+9.034468120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:52.635366 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:52.635296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:52.635366 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:52.635364 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:52.635637 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.635603 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:33:52.635702 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.635625 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:33:52.635702 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.635655 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:52.635819 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.635720 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:56.635701573 +0000 UTC m=+9.135481295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:52.636222 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.636197 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:52.636338 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:52.636256 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:33:56.636238742 +0000 UTC m=+9.136018445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:53.060456 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:53.059998 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:53.060456 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:53.060119 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:33:54.061335 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:54.060565 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:54.061335 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:54.060714 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:33:54.061335 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:54.061169 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:54.061335 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:54.061279 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:33:55.058212 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:55.058179 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:55.058370 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:55.058300 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:33:56.057858 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:56.057809 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:56.058246 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.057951 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:33:56.058246 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:56.058008 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:56.058246 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.058105 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:33:56.574086 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:56.574044 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:56.574309 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.574254 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:56.574394 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.574348 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:34:04.574323419 +0000 UTC m=+17.074103158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:33:56.674901 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:56.674863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:56.675088 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:56.674919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:56.675088 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.675057 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:33:56.675088 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.675079 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:33:56.675249 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.675092 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:56.675249 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.675153 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:04.67513166 +0000 UTC m=+17.174911364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:33:56.675538 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.675512 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:56.675676 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:56.675570 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:04.675554637 +0000 UTC m=+17.175334352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:33:57.058744 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:57.058599 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:57.059166 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:57.058937 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:33:58.059291 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:58.059253 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:33:58.059727 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:58.059302 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:33:58.059727 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:58.059386 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:33:58.059727 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:58.059524 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:33:59.058154 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:33:59.057945 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:33:59.058357 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:33:59.058263 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:00.061776 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:00.061505 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:00.061776 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:00.061629 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:00.061776 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:00.061658 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:00.061776 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:00.061742 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:01.057750 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:01.057717 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:01.057919 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:01.057857 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:02.058040 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:02.058003 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:02.058429 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:02.058003 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:02.058429 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:02.058110 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:02.058429 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:02.058232 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:03.058378 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:03.058346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:03.058813 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:03.058474 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:04.061560 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:04.061526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:04.061989 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.061670 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:04.061989 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:04.061715 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:04.061989 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.061847 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:04.640110 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:04.640075 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:04.640257 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.640230 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:34:04.640310 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.640292 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:34:20.640274816 +0000 UTC m=+33.140054537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:34:04.741010 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:04.740971 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:04.741010 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:04.741009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:04.741240 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.741149 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:34:04.741240 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.741228 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:20.741206026 +0000 UTC m=+33.240985730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:34:04.741328 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.741154 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:34:04.741328 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.741263 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:34:04.741328 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.741277 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:34:04.741328 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:04.741320 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:20.741309016 +0000 UTC m=+33.241088718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:34:05.058543 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:05.058456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:05.058718 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:05.058581 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:06.057870 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:06.057839 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:06.058307 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:06.057959 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:06.058370 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:06.058016 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:06.058547 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:06.058505 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:07.058279 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:07.058243 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:07.058679 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:07.058373 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:08.060810 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:08.060778 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:08.061222 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:08.060816 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:08.061222 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:08.060912 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:08.061222 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:08.061031 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:09.058494 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:09.058452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:09.058676 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:09.058614 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:10.058659 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:10.058619 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:10.059117 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:10.058619 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:10.059117 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:10.058765 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:10.059117 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:10.058846 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:11.058615 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.058456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:11.058748 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:11.058696 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:11.253114 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.253082 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nh469" event={"ID":"064068a9-6858-474d-a457-b2bc3fb6c435","Type":"ContainerStarted","Data":"86e31fb370c9ac4cd81e9073a99863e01cc76bf3b8d5287add59edd4b4c87d0b"} Jul 08 17:34:11.255259 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.255241 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:34:11.255616 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.255594 2574 generic.go:356] "Generic (PLEG): container finished" podID="3655e63d-2e5a-408d-8af6-b1b312194629" containerID="2416883d46b4f8e662818812a03e6721475ebe2811d24aa964d8e7d5d3ecd10f" exitCode=1 Jul 08 17:34:11.255690 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.255673 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"9b58eb0b3c6ed19c750d640eef4537fc49d9647563773880a56f5b13dee79f75"} Jul 08 17:34:11.255757 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.255742 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"1e790b3ca1b0994f7f7d83ad928dd3017d0c14151eb6efabb65e6617abdce58e"} Jul 08 17:34:11.255804 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.255792 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerDied","Data":"2416883d46b4f8e662818812a03e6721475ebe2811d24aa964d8e7d5d3ecd10f"} Jul 08 17:34:11.255862 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.255844 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"7dd1c18c13cef0c3750dce2c9f1eab4fd5a08b8848cb9805aa887334010bc047"} Jul 08 17:34:11.257087 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.257035 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" event={"ID":"57192b8ecd12bde2baff3a89b0094a6a","Type":"ContainerStarted","Data":"3cb20854197817fc6fa6487e227d3cc883aed8f206fd2609aa4ccb4ff0bfe3ec"} Jul 08 17:34:11.258414 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.258395 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-75qzd" event={"ID":"ac6903ba-e074-434e-bb41-2f93a0c29d63","Type":"ContainerStarted","Data":"4e419e11acaaea0fdf6cc404a9ebc4d88372d7c6a7e5a0e6b7a5e367dca825a4"} Jul 08 17:34:11.273078 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.272986 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nh469" podStartSLOduration=2.2532015530000002 podStartE2EDuration="23.272967999s" totalImagesPullingTime="21.019766446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:11.272534492 +0000 UTC m=+23.772314232" watchObservedRunningTime="2026-07-08 17:34:11.272967999 +0000 UTC m=+23.772747724" Jul 08 17:34:11.294008 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.293972 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-115.ec2.internal" podStartSLOduration=22.293958721 podStartE2EDuration="22.293958721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:11.293260967 +0000 UTC m=+23.793040718" watchObservedRunningTime="2026-07-08 17:34:11.293958721 +0000 UTC m=+23.793738444" Jul 08 17:34:11.314705 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:11.314650 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-75qzd" podStartSLOduration=2.428232403 podStartE2EDuration="23.314631588s" totalImagesPullingTime="20.886399185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:11.314270735 +0000 UTC m=+23.814050458" watchObservedRunningTime="2026-07-08 17:34:11.314631588 +0000 UTC m=+23.814411314" Jul 08 17:34:12.058205 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.058000 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:12.058379 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.058056 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:12.058379 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:12.058310 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:12.058512 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:12.058442 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:12.262488 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.262448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s5tj7" event={"ID":"744b8ce9-1887-478f-a460-b648e27cc580","Type":"ContainerStarted","Data":"4425bad020572b97d311ad9fd40d54d0c1a18bbcee2f597c8afe8a40777b9d17"} Jul 08 17:34:12.264146 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.264117 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lq2sd" event={"ID":"eae2679f-b62d-406a-bf32-f5130af69481","Type":"ContainerStarted","Data":"1bb9af3d28722615b27bc4b93a71e4cc7d23adc975834abe24a1be30c4d3b896"} Jul 08 17:34:12.265543 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.265519 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rk9qs" event={"ID":"af2773a3-d6ec-4fd1-8bf1-81440f6c11cb","Type":"ContainerStarted","Data":"bf72f29cf34317ec5ebfc6f300292e8b2011fa765e8348bee15d2f4a5bb4b892"} Jul 08 17:34:12.267160 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.267136 2574 generic.go:356] "Generic (PLEG): container finished" podID="0ecc90ad-bc13-4f07-9e16-a25dd51d9331" containerID="194874a4ea6c929bc5deb9da940e1ce77e6c4ae3d251e2f4e7ba73ab3eb7c648" exitCode=0 Jul 08 17:34:12.267253 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.267230 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerDied","Data":"194874a4ea6c929bc5deb9da940e1ce77e6c4ae3d251e2f4e7ba73ab3eb7c648"} Jul 08 17:34:12.270263 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.270243 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:34:12.270692 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.270669 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"95e3362e3aa5ea9db746e48555aad8b1ca749441e7278003ceffee6f0a86cd51"} Jul 08 17:34:12.270789 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.270731 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"12eed31a69c563dcdf0dbcac18245993185213315891922dd0c055aa7fdb03f0"} Jul 08 17:34:12.272288 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.272266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" event={"ID":"f79f10bb-b4af-43ee-a238-3c9e8949b116","Type":"ContainerStarted","Data":"4baea19af28654e239e4cb01364192cd8455de7ffdf0659f0393f6324f27d9ea"} Jul 08 17:34:12.273789 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.273767 2574 generic.go:356] "Generic (PLEG): container finished" podID="385ef6a0a0ac9ab0104bcd0e7cc3e5b2" containerID="9341444d0106d587aaa61073e441dc821f79146581588f4d13bbcdf01c1aa8da" exitCode=0 Jul 08 17:34:12.273892 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.273855 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" event={"ID":"385ef6a0a0ac9ab0104bcd0e7cc3e5b2","Type":"ContainerDied","Data":"9341444d0106d587aaa61073e441dc821f79146581588f4d13bbcdf01c1aa8da"} Jul 08 17:34:12.275356 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.275328 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zv4d4" event={"ID":"3b713581-56c8-46be-bc7c-51279b58448c","Type":"ContainerStarted","Data":"e9f4353dbe6127c7122983d09e0549debeb0d56f769d7a45ea36d98d93e90064"} Jul 08 17:34:12.303801 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.303754 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s5tj7" podStartSLOduration=3.491555416 podStartE2EDuration="24.303738601s" totalImagesPullingTime="20.812183185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:12.279630537 +0000 UTC m=+24.779410281" watchObservedRunningTime="2026-07-08 17:34:12.303738601 +0000 UTC m=+24.803518324" Jul 08 17:34:12.317077 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.316989 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rk9qs" podStartSLOduration=3.444012845 podStartE2EDuration="24.316970022s" totalImagesPullingTime="20.872957177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:12.316257497 +0000 UTC m=+24.816037221" watchObservedRunningTime="2026-07-08 17:34:12.316970022 +0000 UTC m=+24.816749746" Jul 08 17:34:12.331340 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.331297 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lq2sd" podStartSLOduration=3.3416509469999998 podStartE2EDuration="24.331282754s" totalImagesPullingTime="20.989631807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:12.330501861 +0000 UTC m=+24.830281584" watchObservedRunningTime="2026-07-08 17:34:12.331282754 +0000 UTC m=+24.831062476" Jul 08 17:34:12.358527 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.358476 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zv4d4" podStartSLOduration=3.991055619 podStartE2EDuration="24.358457732s" totalImagesPullingTime="20.367402113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:12.358118391 +0000 UTC m=+24.857898113" watchObservedRunningTime="2026-07-08 17:34:12.358457732 +0000 UTC m=+24.858237457" Jul 08 17:34:12.738053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.738018 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 17:34:12.963174 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.963022 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T17:34:12.73804962Z","UUID":"3ceaed9a-8c9c-4938-859c-4d4a59a36539","Handler":null,"Name":"","Endpoint":""} Jul 08 17:34:12.964741 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.964711 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 17:34:12.964882 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:12.964755 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 17:34:13.058120 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:13.058084 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:13.058283 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:13.058199 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:13.280563 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:13.280472 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" event={"ID":"f79f10bb-b4af-43ee-a238-3c9e8949b116","Type":"ContainerStarted","Data":"f9c1a1dcebdf5e2dba4c1db39cb9891149af14ae23a7d4ebc3cf3dae007dc2b0"} Jul 08 17:34:13.283937 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:13.283907 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" event={"ID":"385ef6a0a0ac9ab0104bcd0e7cc3e5b2","Type":"ContainerStarted","Data":"e80a521f9522272bafcb959355d662d75d0b8f76601c49ba9b4ece834f9ddb82"} Jul 08 17:34:13.300684 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:13.300055 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-115.ec2.internal" podStartSLOduration=24.300029929 podStartE2EDuration="24.300029929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:13.299461192 +0000 UTC m=+25.799240942" watchObservedRunningTime="2026-07-08 17:34:13.300029929 +0000 UTC m=+25.799809653" Jul 08 17:34:13.907535 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:13.907512 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:34:13.908242 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:13.908221 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:34:14.058267 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.058179 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:14.058507 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:14.058309 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:14.058507 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.058374 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:14.058507 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:14.058489 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:14.288231 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.288205 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:34:14.288701 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.288673 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"fccf57b02e991eee4e11fa232df2410a949018cdfe48019cfc2878cc6bc8ac46"} Jul 08 17:34:14.291030 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.291000 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" event={"ID":"f79f10bb-b4af-43ee-a238-3c9e8949b116","Type":"ContainerStarted","Data":"4c5e54f77fe5b6e8c5c7cc5e476e825bd14b41281341a219317d6e4695398c0c"} Jul 08 17:34:14.291288 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.291267 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:34:14.291817 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.291797 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rk9qs" Jul 08 17:34:14.310196 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:14.310117 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v7fjk" podStartSLOduration=3.216278724 podStartE2EDuration="26.310102416s" totalImagesPullingTime="23.093823692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:14.309773033 +0000 UTC m=+26.809552757" watchObservedRunningTime="2026-07-08 17:34:14.310102416 +0000 UTC m=+26.809882139" Jul 08 17:34:15.058053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:15.058018 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:15.058237 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:15.058156 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:16.061589 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:16.061362 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:16.062035 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:16.061426 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:16.062035 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:16.061647 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:16.062035 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:16.061721 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:17.057994 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.057910 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:17.058158 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:17.058019 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:17.299705 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.299532 2574 generic.go:356] "Generic (PLEG): container finished" podID="0ecc90ad-bc13-4f07-9e16-a25dd51d9331" containerID="ca70b2fb0f793ae26a8d57ad8f863c5fb4e81170b4daa2102b5d66b536a4013a" exitCode=0 Jul 08 17:34:17.300538 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.299612 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerDied","Data":"ca70b2fb0f793ae26a8d57ad8f863c5fb4e81170b4daa2102b5d66b536a4013a"} Jul 08 17:34:17.303459 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.303441 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:34:17.303881 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.303859 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"43ebb78b7ade6a794e3d6cc45c9dd4fddfc5e816ae5f932442a0dc51a62a2106"} Jul 08 17:34:17.304123 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.304098 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:34:17.304262 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.304165 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:34:17.304352 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.304335 2574 scope.go:117] "RemoveContainer" containerID="2416883d46b4f8e662818812a03e6721475ebe2811d24aa964d8e7d5d3ecd10f" Jul 08 17:34:17.320989 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:17.320969 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:34:18.059063 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.059036 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:18.059317 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:18.059125 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:18.059317 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.059220 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:18.059402 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:18.059336 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:18.309449 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.309367 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:34:18.309820 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.309739 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" event={"ID":"3655e63d-2e5a-408d-8af6-b1b312194629","Type":"ContainerStarted","Data":"0ebf55d309b97c2a8c5d302eaf10d6880f3363d4ce8cfc8658299c4fee324c3e"} Jul 08 17:34:18.309980 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.309959 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:34:18.325157 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.325135 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:34:18.338342 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:18.338300 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" podStartSLOduration=8.917436481 podStartE2EDuration="30.338287304s" totalImagesPullingTime="21.420850823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:18.337405708 +0000 UTC m=+30.837185431" watchObservedRunningTime="2026-07-08 17:34:18.338287304 +0000 UTC m=+30.838067026" Jul 08 17:34:19.058506 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:19.058474 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:19.058678 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:19.058568 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:19.314205 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:19.314173 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerStarted","Data":"0a0d2dedd165eb7f10f3ebd0882b495472d07877e1e24bbe2ba2af4c0b64321c"} Jul 08 17:34:20.058293 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.058257 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:20.058489 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.058370 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:20.058489 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.058423 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:20.058601 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.058514 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:20.318317 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.318227 2574 generic.go:356] "Generic (PLEG): container finished" podID="0ecc90ad-bc13-4f07-9e16-a25dd51d9331" containerID="0a0d2dedd165eb7f10f3ebd0882b495472d07877e1e24bbe2ba2af4c0b64321c" exitCode=0 Jul 08 17:34:20.318771 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.318309 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerDied","Data":"0a0d2dedd165eb7f10f3ebd0882b495472d07877e1e24bbe2ba2af4c0b64321c"} Jul 08 17:34:20.655882 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.655844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:20.656022 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.655945 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:34:20.656022 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.656012 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs podName:b815e6c6-8eeb-4e56-9a5d-87b762aae43a nodeName:}" failed. No retries permitted until 2026-07-08 17:34:52.655994319 +0000 UTC m=+65.155774039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs") pod "network-metrics-daemon-jwzwj" (UID: "b815e6c6-8eeb-4e56-9a5d-87b762aae43a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:34:20.757357 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.757312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:20.757542 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:20.757371 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:20.757542 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.757507 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:34:20.757542 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.757522 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:34:20.757704 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.757572 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:34:20.757704 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.757586 2574 projected.go:196] Error preparing data for projected volume kube-api-access-cg7pm for pod openshift-network-diagnostics/network-check-target-sf5mq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:34:20.757704 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.757586 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret podName:42a39394-26a1-4f87-8f51-709a58922414 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:52.757565247 +0000 UTC m=+65.257344968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret") pod "global-pull-secret-syncer-sgg9w" (UID: "42a39394-26a1-4f87-8f51-709a58922414") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:34:20.757704 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:20.757656 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm podName:51529eb7-881c-43cf-8586-5b101426f689 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:52.757639279 +0000 UTC m=+65.257418991 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cg7pm" (UniqueName: "kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm") pod "network-check-target-sf5mq" (UID: "51529eb7-881c-43cf-8586-5b101426f689") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:34:21.058245 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:21.058166 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:21.058373 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:21.058266 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:22.058363 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:22.058329 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:22.058767 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:22.058329 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:22.058767 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:22.058449 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:22.058767 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:22.058530 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:22.325810 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:22.325773 2574 generic.go:356] "Generic (PLEG): container finished" podID="0ecc90ad-bc13-4f07-9e16-a25dd51d9331" containerID="8bd10e098a0e9934979a0bf2d6dd6895a03be38e90af8d93f72014d10cedba05" exitCode=0 Jul 08 17:34:22.325986 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:22.325861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerDied","Data":"8bd10e098a0e9934979a0bf2d6dd6895a03be38e90af8d93f72014d10cedba05"} Jul 08 17:34:23.058456 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:23.058423 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:23.058938 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:23.058574 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:24.058318 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:24.058282 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:24.058509 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:24.058427 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:24.058509 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:24.058488 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:24.058878 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:24.058632 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:25.058095 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:25.058066 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:25.058265 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:25.058174 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:26.058708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:26.058667 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:26.058708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:26.058697 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:26.059304 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:26.058861 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:26.059304 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:26.058930 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:27.057748 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:27.057700 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:27.057952 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:27.057819 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:27.339372 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:27.339342 2574 generic.go:356] "Generic (PLEG): container finished" podID="0ecc90ad-bc13-4f07-9e16-a25dd51d9331" containerID="6d1aef4ab376767b853f37e4e445f540be71766926890fac8847a64fbdde88d6" exitCode=0 Jul 08 17:34:27.339746 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:27.339387 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerDied","Data":"6d1aef4ab376767b853f37e4e445f540be71766926890fac8847a64fbdde88d6"} Jul 08 17:34:28.058789 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:28.058749 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:28.059003 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:28.058936 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:28.059003 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:28.058991 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:28.059145 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:28.059122 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:28.347609 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:28.347575 2574 generic.go:356] "Generic (PLEG): container finished" podID="0ecc90ad-bc13-4f07-9e16-a25dd51d9331" containerID="478c2e8a903baaab0751b2b0c0ac1e765b03002b1cfdff49d627b9277191a641" exitCode=0 Jul 08 17:34:28.348028 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:28.347626 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerDied","Data":"478c2e8a903baaab0751b2b0c0ac1e765b03002b1cfdff49d627b9277191a641"} Jul 08 17:34:29.057922 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:29.057883 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:29.058084 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:29.057984 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:29.353535 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:29.353506 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" event={"ID":"0ecc90ad-bc13-4f07-9e16-a25dd51d9331","Type":"ContainerStarted","Data":"c2e47e31b27ae38d60481bcad40cfb2dbd8855949c6feb05bf4f556d65e96f77"} Jul 08 17:34:29.409174 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:29.409117 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7cfqv" podStartSLOduration=8.883604633000001 podStartE2EDuration="41.409101801s" totalImagesPullingTime="32.525497168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:29.404114672 +0000 UTC m=+41.903894395" watchObservedRunningTime="2026-07-08 17:34:29.409101801 +0000 UTC m=+41.908881524" Jul 08 17:34:30.058518 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:30.058484 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:30.058673 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:30.058600 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:30.058673 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:30.058655 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:30.058771 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:30.058741 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:31.058626 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:31.058593 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:31.059029 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:31.058705 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:32.058047 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.058019 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:32.058191 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:32.058150 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:32.058260 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.058198 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:32.058336 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:32.058316 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:32.215345 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.215307 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sgg9w"] Jul 08 17:34:32.216021 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.215903 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sf5mq"] Jul 08 17:34:32.216021 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.215998 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:32.216132 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:32.216095 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:32.222231 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.222207 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jwzwj"] Jul 08 17:34:32.359421 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.359390 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:32.359596 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:32.359390 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:32.359596 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:32.359489 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:32.359700 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:32.359635 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:34.058801 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:34.058769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:34.059292 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:34.058769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:34.059292 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:34.058894 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:34.059292 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:34.058769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:34.059292 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:34.058936 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:34.059292 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:34.058998 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:36.058647 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.058606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:36.059070 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.058606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:36.059070 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:36.058736 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jwzwj" podUID="b815e6c6-8eeb-4e56-9a5d-87b762aae43a" Jul 08 17:34:36.059070 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:36.058797 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sf5mq" podUID="51529eb7-881c-43cf-8586-5b101426f689" Jul 08 17:34:36.059070 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.058606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:36.059070 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:36.058889 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgg9w" podUID="42a39394-26a1-4f87-8f51-709a58922414" Jul 08 17:34:36.319816 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.319746 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-115.ec2.internal" event="NodeReady" Jul 08 17:34:36.319969 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.319896 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 17:34:36.353989 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.353961 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b4456856f-c7mmm"] Jul 08 17:34:36.366343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.366323 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.370301 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.370154 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 17:34:36.371509 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.370439 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 17:34:36.371509 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.370736 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n999c\"" Jul 08 17:34:36.371509 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.370781 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 17:34:36.372529 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.372482 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b4456856f-c7mmm"] Jul 08 17:34:36.376857 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.376801 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nbcq5"] Jul 08 17:34:36.377251 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.377222 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 17:34:36.398437 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.398404 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dpxz2"] Jul 08 17:34:36.398599 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.398533 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.400880 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.400855 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 17:34:36.401078 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.401010 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 17:34:36.401189 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.401138 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 17:34:36.401273 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.401197 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j9pdx\"" Jul 08 17:34:36.401273 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.401227 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 17:34:36.418288 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.418270 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dpxz2"] Jul 08 17:34:36.418389 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.418298 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nbcq5"] Jul 08 17:34:36.418389 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.418385 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.420608 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.420589 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9p74p\"" Jul 08 17:34:36.420933 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.420917 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 17:34:36.421016 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.421002 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 17:34:36.472925 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.472902 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e0109f3-0220-4fda-a6d8-622efc6c99a1-image-registry-private-configuration\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.472961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e0109f3-0220-4fda-a6d8-622efc6c99a1-registry-certificates\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.472982 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e0109f3-0220-4fda-a6d8-622efc6c99a1-ca-trust-extracted\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.473021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e0109f3-0220-4fda-a6d8-622efc6c99a1-trusted-ca\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.473050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e0109f3-0220-4fda-a6d8-622efc6c99a1-installation-pull-secrets\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.473068 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx2ss\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-kube-api-access-xx2ss\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473264 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.473087 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-bound-sa-token\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.473264 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.473122 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-registry-tls\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.474943 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.474924 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zdwng"] Jul 08 17:34:36.488458 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.488437 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.490773 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.490736 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 17:34:36.490978 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.490965 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 17:34:36.491343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.491325 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wnq9l\"" Jul 08 17:34:36.491517 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.491345 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 17:34:36.492516 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.492498 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zdwng"] Jul 08 17:34:36.573908 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.573886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/508d9609-ed36-456c-9743-8919d19f667d-crio-socket\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.573917 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/508d9609-ed36-456c-9743-8919d19f667d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.573941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e0109f3-0220-4fda-a6d8-622efc6c99a1-registry-certificates\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.573962 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e0109f3-0220-4fda-a6d8-622efc6c99a1-ca-trust-extracted\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.573979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e0109f3-0220-4fda-a6d8-622efc6c99a1-trusted-ca\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.573994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e0109f3-0220-4fda-a6d8-622efc6c99a1-installation-pull-secrets\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx2ss\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-kube-api-access-xx2ss\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574026 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/508d9609-ed36-456c-9743-8919d19f667d-data-volume\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.574060 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574047 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz8qf\" (UniqueName: \"kubernetes.io/projected/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-kube-api-access-kz8qf\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.574413 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574074 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-bound-sa-token\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574413 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574130 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-config-volume\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.574413 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/508d9609-ed36-456c-9743-8919d19f667d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.574413 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574399 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e0109f3-0220-4fda-a6d8-622efc6c99a1-ca-trust-extracted\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574613 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574521 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-registry-tls\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574673 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e0109f3-0220-4fda-a6d8-622efc6c99a1-image-registry-private-configuration\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.574673 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574653 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-metrics-tls\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.574769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-tmp-dir\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.574769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574714 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9644\" (UniqueName: \"kubernetes.io/projected/508d9609-ed36-456c-9743-8919d19f667d-kube-api-access-f9644\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.574880 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.574853 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e0109f3-0220-4fda-a6d8-622efc6c99a1-registry-certificates\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.575121 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.575104 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e0109f3-0220-4fda-a6d8-622efc6c99a1-trusted-ca\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.578085 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.578065 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e0109f3-0220-4fda-a6d8-622efc6c99a1-image-registry-private-configuration\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.578270 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.578253 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-registry-tls\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.578312 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.578253 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e0109f3-0220-4fda-a6d8-622efc6c99a1-installation-pull-secrets\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.582262 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.582232 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-bound-sa-token\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.582469 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.582453 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx2ss\" (UniqueName: \"kubernetes.io/projected/5e0109f3-0220-4fda-a6d8-622efc6c99a1-kube-api-access-xx2ss\") pod \"image-registry-6b4456856f-c7mmm\" (UID: \"5e0109f3-0220-4fda-a6d8-622efc6c99a1\") " pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.675530 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675496 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz8qf\" (UniqueName: \"kubernetes.io/projected/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-kube-api-access-kz8qf\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.675711 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-config-volume\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.675711 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/508d9609-ed36-456c-9743-8919d19f667d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.675711 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675598 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc5f6ad9-9612-4a84-a375-36b8f3916d69-cert\") pod \"ingress-canary-zdwng\" (UID: \"dc5f6ad9-9612-4a84-a375-36b8f3916d69\") " pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.675711 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhhmb\" (UniqueName: \"kubernetes.io/projected/dc5f6ad9-9612-4a84-a375-36b8f3916d69-kube-api-access-xhhmb\") pod \"ingress-canary-zdwng\" (UID: \"dc5f6ad9-9612-4a84-a375-36b8f3916d69\") " pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.675711 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675670 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-metrics-tls\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.675711 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-tmp-dir\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.676068 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9644\" (UniqueName: \"kubernetes.io/projected/508d9609-ed36-456c-9743-8919d19f667d-kube-api-access-f9644\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.676068 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/508d9609-ed36-456c-9743-8919d19f667d-crio-socket\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.676068 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/508d9609-ed36-456c-9743-8919d19f667d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.676068 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.675968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/508d9609-ed36-456c-9743-8919d19f667d-data-volume\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.676264 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.676139 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-tmp-dir\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.676264 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.676216 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/508d9609-ed36-456c-9743-8919d19f667d-crio-socket\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.676367 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.676302 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/508d9609-ed36-456c-9743-8919d19f667d-data-volume\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.676367 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.676327 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-config-volume\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.678122 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.678097 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-metrics-tls\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.678301 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.678282 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/508d9609-ed36-456c-9743-8919d19f667d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.680231 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.680214 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:36.683931 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.683907 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz8qf\" (UniqueName: \"kubernetes.io/projected/e7c9fd72-b1d2-42d5-9374-018fcd1e6776-kube-api-access-kz8qf\") pod \"dns-default-dpxz2\" (UID: \"e7c9fd72-b1d2-42d5-9374-018fcd1e6776\") " pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.686274 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.686253 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/508d9609-ed36-456c-9743-8919d19f667d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.694455 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.694436 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9644\" (UniqueName: \"kubernetes.io/projected/508d9609-ed36-456c-9743-8919d19f667d-kube-api-access-f9644\") pod \"insights-runtime-extractor-nbcq5\" (UID: \"508d9609-ed36-456c-9743-8919d19f667d\") " pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.706784 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.706669 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nbcq5" Jul 08 17:34:36.726708 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.726687 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:36.777891 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.776240 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc5f6ad9-9612-4a84-a375-36b8f3916d69-cert\") pod \"ingress-canary-zdwng\" (UID: \"dc5f6ad9-9612-4a84-a375-36b8f3916d69\") " pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.777891 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.776478 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhmb\" (UniqueName: \"kubernetes.io/projected/dc5f6ad9-9612-4a84-a375-36b8f3916d69-kube-api-access-xhhmb\") pod \"ingress-canary-zdwng\" (UID: \"dc5f6ad9-9612-4a84-a375-36b8f3916d69\") " pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.782676 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.782615 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc5f6ad9-9612-4a84-a375-36b8f3916d69-cert\") pod \"ingress-canary-zdwng\" (UID: \"dc5f6ad9-9612-4a84-a375-36b8f3916d69\") " pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.786422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.786397 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhhmb\" (UniqueName: \"kubernetes.io/projected/dc5f6ad9-9612-4a84-a375-36b8f3916d69-kube-api-access-xhhmb\") pod \"ingress-canary-zdwng\" (UID: \"dc5f6ad9-9612-4a84-a375-36b8f3916d69\") " pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.797877 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.797697 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zdwng" Jul 08 17:34:36.895552 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.891108 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b4456856f-c7mmm"] Jul 08 17:34:36.895552 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.894563 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nbcq5"] Jul 08 17:34:36.900148 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:36.900113 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0109f3_0220_4fda_a6d8_622efc6c99a1.slice/crio-520e20e8c88c8d115c17db81216db29a1879c2e6ec614397a955df513080fb3d WatchSource:0}: Error finding container 520e20e8c88c8d115c17db81216db29a1879c2e6ec614397a955df513080fb3d: Status 404 returned error can't find the container with id 520e20e8c88c8d115c17db81216db29a1879c2e6ec614397a955df513080fb3d Jul 08 17:34:36.901043 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:36.901016 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508d9609_ed36_456c_9743_8919d19f667d.slice/crio-61943192f48dad9982ad543d70e322f8e8c4b8863d8ea73c61e20c4b7bda7eb2 WatchSource:0}: Error finding container 61943192f48dad9982ad543d70e322f8e8c4b8863d8ea73c61e20c4b7bda7eb2: Status 404 returned error can't find the container with id 61943192f48dad9982ad543d70e322f8e8c4b8863d8ea73c61e20c4b7bda7eb2 Jul 08 17:34:36.911787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.911767 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dpxz2"] Jul 08 17:34:36.915874 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:36.915854 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c9fd72_b1d2_42d5_9374_018fcd1e6776.slice/crio-afbc32cc130ee36041056a37dc4d6565718df787bd3a62b8c3cb7f27faf16091 WatchSource:0}: Error finding container afbc32cc130ee36041056a37dc4d6565718df787bd3a62b8c3cb7f27faf16091: Status 404 returned error can't find the container with id afbc32cc130ee36041056a37dc4d6565718df787bd3a62b8c3cb7f27faf16091 Jul 08 17:34:36.966059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:36.966032 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zdwng"] Jul 08 17:34:36.978684 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:36.978658 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5f6ad9_9612_4a84_a375_36b8f3916d69.slice/crio-52e1b0211213e4fb8890aa4c3c2be9cacf9ba071032cc09d93353c0ba0f2d444 WatchSource:0}: Error finding container 52e1b0211213e4fb8890aa4c3c2be9cacf9ba071032cc09d93353c0ba0f2d444: Status 404 returned error can't find the container with id 52e1b0211213e4fb8890aa4c3c2be9cacf9ba071032cc09d93353c0ba0f2d444 Jul 08 17:34:37.375343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.375309 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zdwng" event={"ID":"dc5f6ad9-9612-4a84-a375-36b8f3916d69","Type":"ContainerStarted","Data":"52e1b0211213e4fb8890aa4c3c2be9cacf9ba071032cc09d93353c0ba0f2d444"} Jul 08 17:34:37.376554 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.376522 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbcq5" event={"ID":"508d9609-ed36-456c-9743-8919d19f667d","Type":"ContainerStarted","Data":"61943192f48dad9982ad543d70e322f8e8c4b8863d8ea73c61e20c4b7bda7eb2"} Jul 08 17:34:37.378197 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.378171 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" event={"ID":"5e0109f3-0220-4fda-a6d8-622efc6c99a1","Type":"ContainerStarted","Data":"53b2788714bbcb1c44ad84a4cc6db6b2ddd72b69339622b8f35e644ceb0606d4"} Jul 08 17:34:37.378316 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.378215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" event={"ID":"5e0109f3-0220-4fda-a6d8-622efc6c99a1","Type":"ContainerStarted","Data":"520e20e8c88c8d115c17db81216db29a1879c2e6ec614397a955df513080fb3d"} Jul 08 17:34:37.378316 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.378240 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:37.379578 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.379555 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dpxz2" event={"ID":"e7c9fd72-b1d2-42d5-9374-018fcd1e6776","Type":"ContainerStarted","Data":"afbc32cc130ee36041056a37dc4d6565718df787bd3a62b8c3cb7f27faf16091"} Jul 08 17:34:37.397933 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:37.397887 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" podStartSLOduration=6.397872645 podStartE2EDuration="6.397872645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:37.397778593 +0000 UTC m=+49.897558326" watchObservedRunningTime="2026-07-08 17:34:37.397872645 +0000 UTC m=+49.897652368" Jul 08 17:34:38.063168 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.063115 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:38.063322 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.063266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:38.063322 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.063296 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:38.065938 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.065917 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:34:38.066539 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.066515 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4jhm\"" Jul 08 17:34:38.066539 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.066537 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:34:38.066877 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.066806 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:34:38.066965 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.066912 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ktdt9\"" Jul 08 17:34:38.066965 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:38.066942 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:34:39.389076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:39.388745 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dpxz2" event={"ID":"e7c9fd72-b1d2-42d5-9374-018fcd1e6776","Type":"ContainerStarted","Data":"7e62d7dcd44ac2d810543abf14fec6cc4c0274ec24e014b3e2fd35119873a25b"} Jul 08 17:34:39.389076 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:39.389037 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dpxz2" event={"ID":"e7c9fd72-b1d2-42d5-9374-018fcd1e6776","Type":"ContainerStarted","Data":"956c5f465043bb921b652b635fbb112346826f8f107b84b7fbb0c8ebdc713f82"} Jul 08 17:34:40.391911 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:40.391881 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:40.414323 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:40.414277 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dpxz2" podStartSLOduration=2.321795939 podStartE2EDuration="4.414259352s" totalImagesPullingTime="2.092463413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:40.412813105 +0000 UTC m=+52.912592859" watchObservedRunningTime="2026-07-08 17:34:40.414259352 +0000 UTC m=+52.914039077" Jul 08 17:34:43.401571 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:43.401543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zdwng" event={"ID":"dc5f6ad9-9612-4a84-a375-36b8f3916d69","Type":"ContainerStarted","Data":"3e7e05eeb734045a4f35d50ff5acc7bffe9b1abfffbae210b4c11b867944d211"} Jul 08 17:34:43.415780 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:43.415732 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zdwng" podStartSLOduration=1.075495493 podStartE2EDuration="7.415715975s" totalImagesPullingTime="6.340220482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:43.415377986 +0000 UTC m=+55.915157708" watchObservedRunningTime="2026-07-08 17:34:43.415715975 +0000 UTC m=+55.915495697" Jul 08 17:34:44.406442 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:44.406254 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbcq5" event={"ID":"508d9609-ed36-456c-9743-8919d19f667d","Type":"ContainerStarted","Data":"c85d72c40a7f8d86bc75c4623224408341eeff66d49279d3b7433c1301576ed3"} Jul 08 17:34:46.413932 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:46.413889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbcq5" event={"ID":"508d9609-ed36-456c-9743-8919d19f667d","Type":"ContainerStarted","Data":"8c7925cc1b1bcd429693826ded0c7f6f400a551da5aa3ffb75e7c400469dabed"} Jul 08 17:34:46.998905 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:46.998859 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw"] Jul 08 17:34:47.001887 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.001863 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.007402 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.007373 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 17:34:47.007578 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.007548 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 17:34:47.008702 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.008675 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 17:34:47.009023 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.009007 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 17:34:47.009262 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.009246 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-96x85\"" Jul 08 17:34:47.009358 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.009341 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:34:47.020429 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.020405 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw"] Jul 08 17:34:47.044221 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.044183 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k"] Jul 08 17:34:47.046384 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.046356 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.050198 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.049874 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 17:34:47.050198 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.049996 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m52t9\"" Jul 08 17:34:47.050198 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.050048 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:34:47.050355 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.050333 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 17:34:47.055292 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.055271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.055397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.055326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.055397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.055382 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.055510 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.055405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-475fv\" (UniqueName: \"kubernetes.io/projected/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-kube-api-access-475fv\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.063354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.063333 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gdvfv"] Jul 08 17:34:47.065285 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.065270 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.068025 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.067997 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 17:34:47.068202 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.068115 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 17:34:47.068281 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.068273 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 17:34:47.068744 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.068616 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-glrk5\"" Jul 08 17:34:47.069358 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.069335 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k"] Jul 08 17:34:47.155926 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.155895 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-475fv\" (UniqueName: \"kubernetes.io/projected/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-kube-api-access-475fv\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.156106 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.155937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.156106 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.155962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-tls\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156106 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156013 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-root\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156106 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156047 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.156106 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156096 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0026793-f06e-44cb-8d51-6aa993a0bd83-metrics-client-ca\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbm4j\" (UniqueName: \"kubernetes.io/projected/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-api-access-vbm4j\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156180 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k5xs\" (UniqueName: \"kubernetes.io/projected/e0026793-f06e-44cb-8d51-6aa993a0bd83-kube-api-access-8k5xs\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac4bc77c-7758-4320-9947-3ba2a4cae049-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156267 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156301 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156343 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac4bc77c-7758-4320-9947-3ba2a4cae049-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-sys\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156393 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156423 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:47.156435 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156453 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-wtmp\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:47.156491 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-tls podName:6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:47.656472495 +0000 UTC m=+60.156252220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-pd4nw" (UID: "6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9") : secret "openshift-state-metrics-tls" not found Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.156657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.156562 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-textfile\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.157243 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.157213 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.160851 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.160807 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.167427 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.167403 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-475fv\" (UniqueName: \"kubernetes.io/projected/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-kube-api-access-475fv\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.257144 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257054 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k5xs\" (UniqueName: \"kubernetes.io/projected/e0026793-f06e-44cb-8d51-6aa993a0bd83-kube-api-access-8k5xs\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257144 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257113 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.257366 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac4bc77c-7758-4320-9947-3ba2a4cae049-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.257366 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257366 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac4bc77c-7758-4320-9947-3ba2a4cae049-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.257366 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-sys\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-wtmp\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-textfile\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257467 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-sys\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257481 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257508 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ac4bc77c-7758-4320-9947-3ba2a4cae049-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257525 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-tls\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257552 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-root\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.257575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257579 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257640 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0026793-f06e-44cb-8d51-6aa993a0bd83-metrics-client-ca\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-wtmp\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbm4j\" (UniqueName: \"kubernetes.io/projected/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-api-access-vbm4j\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.257692 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0026793-f06e-44cb-8d51-6aa993a0bd83-root\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:47.257776 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: E0708 17:34:47.257859 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-tls podName:e0026793-f06e-44cb-8d51-6aa993a0bd83 nodeName:}" failed. No retries permitted until 2026-07-08 17:34:47.757815315 +0000 UTC m=+60.257595032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-tls") pod "node-exporter-gdvfv" (UID: "e0026793-f06e-44cb-8d51-6aa993a0bd83") : secret "node-exporter-tls" not found Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.258100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-accelerators-collector-config\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.258163 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-textfile\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.258207 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.258185 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac4bc77c-7758-4320-9947-3ba2a4cae049-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.258671 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.258408 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0026793-f06e-44cb-8d51-6aa993a0bd83-metrics-client-ca\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.258671 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.258655 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.260202 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.260177 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.260415 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.260398 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.260611 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.260592 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.267206 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.267187 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k5xs\" (UniqueName: \"kubernetes.io/projected/e0026793-f06e-44cb-8d51-6aa993a0bd83-kube-api-access-8k5xs\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.267667 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.267643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbm4j\" (UniqueName: \"kubernetes.io/projected/ac4bc77c-7758-4320-9947-3ba2a4cae049-kube-api-access-vbm4j\") pod \"kube-state-metrics-7c9dbb8b7d-7lw7k\" (UID: \"ac4bc77c-7758-4320-9947-3ba2a4cae049\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.356313 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.356266 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" Jul 08 17:34:47.662041 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.662008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.664923 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.664884 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pd4nw\" (UID: \"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.762583 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.762538 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-tls\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.765072 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.765046 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0026793-f06e-44cb-8d51-6aa993a0bd83-node-exporter-tls\") pod \"node-exporter-gdvfv\" (UID: \"e0026793-f06e-44cb-8d51-6aa993a0bd83\") " pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.920561 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.920326 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-96x85\"" Jul 08 17:34:47.922841 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.922647 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" Jul 08 17:34:47.977753 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.977732 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-glrk5\"" Jul 08 17:34:47.986132 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:47.986104 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gdvfv" Jul 08 17:34:47.997103 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:47.997072 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0026793_f06e_44cb_8d51_6aa993a0bd83.slice/crio-db9d46f1222c46d4fe891661869decbce38cb49e0d28696ecb0353c7fc0bc8d3 WatchSource:0}: Error finding container db9d46f1222c46d4fe891661869decbce38cb49e0d28696ecb0353c7fc0bc8d3: Status 404 returned error can't find the container with id db9d46f1222c46d4fe891661869decbce38cb49e0d28696ecb0353c7fc0bc8d3 Jul 08 17:34:48.012340 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.012316 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k"] Jul 08 17:34:48.064601 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.064564 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw"] Jul 08 17:34:48.066971 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:48.066945 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ead4133_1c89_4d01_b1b4_52a4f7a1b5e9.slice/crio-f747ae4be47313f6719b6ce338a0a4716f780a17573042bb7e5e2cc8a2fe6acb WatchSource:0}: Error finding container f747ae4be47313f6719b6ce338a0a4716f780a17573042bb7e5e2cc8a2fe6acb: Status 404 returned error can't find the container with id f747ae4be47313f6719b6ce338a0a4716f780a17573042bb7e5e2cc8a2fe6acb Jul 08 17:34:48.422584 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.422547 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdvfv" event={"ID":"e0026793-f06e-44cb-8d51-6aa993a0bd83","Type":"ContainerStarted","Data":"db9d46f1222c46d4fe891661869decbce38cb49e0d28696ecb0353c7fc0bc8d3"} Jul 08 17:34:48.424636 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.424599 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nbcq5" event={"ID":"508d9609-ed36-456c-9743-8919d19f667d","Type":"ContainerStarted","Data":"e3c66171510832ba657d9e00abe80cb011021114a46136d5b0ab88f220c20867"} Jul 08 17:34:48.426432 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.426408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" event={"ID":"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9","Type":"ContainerStarted","Data":"41686e946b6c85ab1d56d7892842ba67eafb7f5894c9b7668dca21011ef337b1"} Jul 08 17:34:48.426532 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.426460 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" event={"ID":"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9","Type":"ContainerStarted","Data":"933f08f46209b24868c231998703bdd029185f2e43450cf387adf39f71bdd526"} Jul 08 17:34:48.426532 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.426489 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" event={"ID":"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9","Type":"ContainerStarted","Data":"f747ae4be47313f6719b6ce338a0a4716f780a17573042bb7e5e2cc8a2fe6acb"} Jul 08 17:34:48.427425 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.427407 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" event={"ID":"ac4bc77c-7758-4320-9947-3ba2a4cae049","Type":"ContainerStarted","Data":"33fe55f0ab409fe7a8f4065ecc9eaf7e74e02131a750fb48d7c4023fd0eebd22"} Jul 08 17:34:48.443423 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:48.443375 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nbcq5" podStartSLOduration=1.582121731 podStartE2EDuration="12.443360588s" totalImagesPullingTime="10.861238857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:34:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:48.442040149 +0000 UTC m=+60.941819900" watchObservedRunningTime="2026-07-08 17:34:48.443360588 +0000 UTC m=+60.943140310" Jul 08 17:34:50.336506 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:50.336477 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt95p" Jul 08 17:34:50.396629 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:50.396608 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dpxz2" Jul 08 17:34:50.437382 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:50.437347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" event={"ID":"ac4bc77c-7758-4320-9947-3ba2a4cae049","Type":"ContainerStarted","Data":"b23c532b9c7f22c8555c7b9649cbc57974e54380f88e2ba362972920db3bcf41"} Jul 08 17:34:51.376880 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.376850 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-cc46778b7-j2cw4"] Jul 08 17:34:51.378717 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.378675 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.381519 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.381491 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 17:34:51.381519 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.381507 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 17:34:51.381675 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.381521 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 17:34:51.381675 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.381509 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 17:34:51.382299 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.382283 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pwncn\"" Jul 08 17:34:51.382396 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.382319 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4u5be01h1oisv\"" Jul 08 17:34:51.393445 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.393424 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-cc46778b7-j2cw4"] Jul 08 17:34:51.442719 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.442681 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" event={"ID":"6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9","Type":"ContainerStarted","Data":"cad12fbebe0f548fdc286a02d97765990c88b3ac00f57bb4fc40d62c5fc00948"} Jul 08 17:34:51.444710 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.444687 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" event={"ID":"ac4bc77c-7758-4320-9947-3ba2a4cae049","Type":"ContainerStarted","Data":"404ec7061cceff3ebeb2e95cef27c26a3a1f7fb3ea15c0b7218c4b8ed28bacce"} Jul 08 17:34:51.444814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.444738 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" event={"ID":"ac4bc77c-7758-4320-9947-3ba2a4cae049","Type":"ContainerStarted","Data":"e3f285a7f928b76fabbbfc05e30684a7cdf6178badc1eaaa770a9e97c01fec1e"} Jul 08 17:34:51.446253 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.446233 2574 generic.go:356] "Generic (PLEG): container finished" podID="e0026793-f06e-44cb-8d51-6aa993a0bd83" containerID="1b3227f0439cc58a722fe393d2165d5d33bfbff22e1b3ae6b13cc2f83cf9a809" exitCode=0 Jul 08 17:34:51.446353 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.446265 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdvfv" event={"ID":"e0026793-f06e-44cb-8d51-6aa993a0bd83","Type":"ContainerDied","Data":"1b3227f0439cc58a722fe393d2165d5d33bfbff22e1b3ae6b13cc2f83cf9a809"} Jul 08 17:34:51.475661 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.475613 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pd4nw" podStartSLOduration=2.594929875 podStartE2EDuration="5.47559476s" totalImagesPullingTime="2.880664885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:51.475036095 +0000 UTC m=+63.974815834" watchObservedRunningTime="2026-07-08 17:34:51.47559476 +0000 UTC m=+63.975374482" Jul 08 17:34:51.492874 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.492853 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/71be1e72-bd7e-4498-989b-bfd5f7328886-audit-log\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.492969 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.492916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-secret-metrics-server-tls\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.493471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.493168 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mhn2\" (UniqueName: \"kubernetes.io/projected/71be1e72-bd7e-4498-989b-bfd5f7328886-kube-api-access-9mhn2\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.493471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.493223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-secret-metrics-server-client-certs\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.493471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.493270 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-client-ca-bundle\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.493658 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.493517 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/71be1e72-bd7e-4498-989b-bfd5f7328886-metrics-server-audit-profiles\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.497552 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.493570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71be1e72-bd7e-4498-989b-bfd5f7328886-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.505869 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.505815 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7lw7k" podStartSLOduration=2.624997147 podStartE2EDuration="4.505802135s" totalImagesPullingTime="1.880804988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:51.504423424 +0000 UTC m=+64.004203180" watchObservedRunningTime="2026-07-08 17:34:51.505802135 +0000 UTC m=+64.005581858" Jul 08 17:34:51.598652 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mhn2\" (UniqueName: \"kubernetes.io/projected/71be1e72-bd7e-4498-989b-bfd5f7328886-kube-api-access-9mhn2\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.598773 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-secret-metrics-server-client-certs\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.598773 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598688 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-client-ca-bundle\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.598773 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/71be1e72-bd7e-4498-989b-bfd5f7328886-metrics-server-audit-profiles\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.598773 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71be1e72-bd7e-4498-989b-bfd5f7328886-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.598990 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598800 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/71be1e72-bd7e-4498-989b-bfd5f7328886-audit-log\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.598990 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.598853 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-secret-metrics-server-tls\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.599304 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.599272 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/71be1e72-bd7e-4498-989b-bfd5f7328886-audit-log\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.599932 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.599904 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71be1e72-bd7e-4498-989b-bfd5f7328886-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.600118 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.600088 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/71be1e72-bd7e-4498-989b-bfd5f7328886-metrics-server-audit-profiles\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.601398 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.601376 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-secret-metrics-server-client-certs\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.601485 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.601426 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-secret-metrics-server-tls\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.601584 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.601562 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71be1e72-bd7e-4498-989b-bfd5f7328886-client-ca-bundle\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.608116 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.608091 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mhn2\" (UniqueName: \"kubernetes.io/projected/71be1e72-bd7e-4498-989b-bfd5f7328886-kube-api-access-9mhn2\") pod \"metrics-server-cc46778b7-j2cw4\" (UID: \"71be1e72-bd7e-4498-989b-bfd5f7328886\") " pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.687902 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.687793 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:34:51.820597 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:51.820568 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-cc46778b7-j2cw4"] Jul 08 17:34:51.823425 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:51.823398 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71be1e72_bd7e_4498_989b_bfd5f7328886.slice/crio-d9967b0c4ec22c0ef7da964d500bb78b5daa72739b96e28864096936bb37b19e WatchSource:0}: Error finding container d9967b0c4ec22c0ef7da964d500bb78b5daa72739b96e28864096936bb37b19e: Status 404 returned error can't find the container with id d9967b0c4ec22c0ef7da964d500bb78b5daa72739b96e28864096936bb37b19e Jul 08 17:34:52.450809 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.450769 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" event={"ID":"71be1e72-bd7e-4498-989b-bfd5f7328886","Type":"ContainerStarted","Data":"d9967b0c4ec22c0ef7da964d500bb78b5daa72739b96e28864096936bb37b19e"} Jul 08 17:34:52.452902 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.452866 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdvfv" event={"ID":"e0026793-f06e-44cb-8d51-6aa993a0bd83","Type":"ContainerStarted","Data":"b91638bb22bdf66f3d98245e20b71a0635f7065fa0292605907ca584de636b4a"} Jul 08 17:34:52.453026 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.452927 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gdvfv" event={"ID":"e0026793-f06e-44cb-8d51-6aa993a0bd83","Type":"ContainerStarted","Data":"39a194007582fa77d98adc5d10259a0b93cba60a5ddc0100f724466a47f44ba6"} Jul 08 17:34:52.481690 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.481643 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gdvfv" podStartSLOduration=2.414643417 podStartE2EDuration="5.481626731s" totalImagesPullingTime="2.99069096s" totalInitContainerRuntime="76.292354ms" isStatefulPod=true podCreationTimestamp="2026-07-08 17:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:52.480658089 +0000 UTC m=+64.980437840" watchObservedRunningTime="2026-07-08 17:34:52.481626731 +0000 UTC m=+64.981406454" Jul 08 17:34:52.707427 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.707317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:52.709725 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.709702 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:34:52.720103 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.720079 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b815e6c6-8eeb-4e56-9a5d-87b762aae43a-metrics-certs\") pod \"network-metrics-daemon-jwzwj\" (UID: \"b815e6c6-8eeb-4e56-9a5d-87b762aae43a\") " pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:52.778948 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.778913 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4jhm\"" Jul 08 17:34:52.787020 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.786998 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jwzwj" Jul 08 17:34:52.808566 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.808535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:52.808681 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.808618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:52.811753 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.811730 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:34:52.811885 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.811764 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:34:52.822495 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.821132 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:34:52.822495 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.821587 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/42a39394-26a1-4f87-8f51-709a58922414-original-pull-secret\") pod \"global-pull-secret-syncer-sgg9w\" (UID: \"42a39394-26a1-4f87-8f51-709a58922414\") " pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:52.831912 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.831884 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg7pm\" (UniqueName: \"kubernetes.io/projected/51529eb7-881c-43cf-8586-5b101426f689-kube-api-access-cg7pm\") pod \"network-check-target-sf5mq\" (UID: \"51529eb7-881c-43cf-8586-5b101426f689\") " pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:52.932046 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:52.932009 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jwzwj"] Jul 08 17:34:52.935931 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:52.935899 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb815e6c6_8eeb_4e56_9a5d_87b762aae43a.slice/crio-09a4eafc9b04ea30b7af5f41cdb376bdda5c4b188760adb45ef54704f27b839f WatchSource:0}: Error finding container 09a4eafc9b04ea30b7af5f41cdb376bdda5c4b188760adb45ef54704f27b839f: Status 404 returned error can't find the container with id 09a4eafc9b04ea30b7af5f41cdb376bdda5c4b188760adb45ef54704f27b839f Jul 08 17:34:53.085248 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:53.085207 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgg9w" Jul 08 17:34:53.100228 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:53.100202 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ktdt9\"" Jul 08 17:34:53.107850 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:53.107808 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:53.249968 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:53.249938 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sgg9w"] Jul 08 17:34:53.263843 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:53.263789 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sf5mq"] Jul 08 17:34:53.458537 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:53.458441 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jwzwj" event={"ID":"b815e6c6-8eeb-4e56-9a5d-87b762aae43a","Type":"ContainerStarted","Data":"09a4eafc9b04ea30b7af5f41cdb376bdda5c4b188760adb45ef54704f27b839f"} Jul 08 17:34:53.589782 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:53.589740 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a39394_26a1_4f87_8f51_709a58922414.slice/crio-5e33abfe849cd2820fd16c83f277691734658cb2135925732b54d99f9695290e WatchSource:0}: Error finding container 5e33abfe849cd2820fd16c83f277691734658cb2135925732b54d99f9695290e: Status 404 returned error can't find the container with id 5e33abfe849cd2820fd16c83f277691734658cb2135925732b54d99f9695290e Jul 08 17:34:53.590324 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:34:53.590270 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51529eb7_881c_43cf_8586_5b101426f689.slice/crio-af586ef4e2184df4599bb417dc8441a6c84213c0a8ca868fd09bd169da43393c WatchSource:0}: Error finding container af586ef4e2184df4599bb417dc8441a6c84213c0a8ca868fd09bd169da43393c: Status 404 returned error can't find the container with id af586ef4e2184df4599bb417dc8441a6c84213c0a8ca868fd09bd169da43393c Jul 08 17:34:54.465000 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:54.464966 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" event={"ID":"71be1e72-bd7e-4498-989b-bfd5f7328886","Type":"ContainerStarted","Data":"4b10acf3187e8d476767cd6a8f97b8ea8e60c93c1001fe7bb8d17a0162ed8fc9"} Jul 08 17:34:54.466554 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:54.466519 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sf5mq" event={"ID":"51529eb7-881c-43cf-8586-5b101426f689","Type":"ContainerStarted","Data":"af586ef4e2184df4599bb417dc8441a6c84213c0a8ca868fd09bd169da43393c"} Jul 08 17:34:54.468276 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:54.468251 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jwzwj" event={"ID":"b815e6c6-8eeb-4e56-9a5d-87b762aae43a","Type":"ContainerStarted","Data":"521923a3496e84d17559c49c439843ebc90360bea0f9fe961b4064ecd8584467"} Jul 08 17:34:54.470084 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:54.470059 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sgg9w" event={"ID":"42a39394-26a1-4f87-8f51-709a58922414","Type":"ContainerStarted","Data":"5e33abfe849cd2820fd16c83f277691734658cb2135925732b54d99f9695290e"} Jul 08 17:34:54.484344 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:54.484305 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" podStartSLOduration=1.695163252 podStartE2EDuration="3.484291626s" totalImagesPullingTime="1.789128374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:54.482988317 +0000 UTC m=+66.982768056" watchObservedRunningTime="2026-07-08 17:34:54.484291626 +0000 UTC m=+66.984071349" Jul 08 17:34:55.477729 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:55.477689 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jwzwj" event={"ID":"b815e6c6-8eeb-4e56-9a5d-87b762aae43a","Type":"ContainerStarted","Data":"7abe451dcd38d9ad20f0f87d098af911974a5d87066c2e5f6386058dc0d458f0"} Jul 08 17:34:55.500182 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:55.500125 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jwzwj" podStartSLOduration=66.150046768 podStartE2EDuration="1m7.500107835s" totalImagesPullingTime="1.350061067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:55.499041981 +0000 UTC m=+67.998821705" watchObservedRunningTime="2026-07-08 17:34:55.500107835 +0000 UTC m=+67.999887557" Jul 08 17:34:58.388618 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:58.388581 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b4456856f-c7mmm" Jul 08 17:34:59.494877 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:59.494845 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sf5mq" event={"ID":"51529eb7-881c-43cf-8586-5b101426f689","Type":"ContainerStarted","Data":"7b3d4ee7ab6ff0e23b2aa1bed18cd245f50d24d4d4344332c54fdc14512a461c"} Jul 08 17:34:59.495336 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:59.494973 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:34:59.496284 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:59.496256 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sgg9w" event={"ID":"42a39394-26a1-4f87-8f51-709a58922414","Type":"ContainerStarted","Data":"c760a16b085cff60d63da8b28e0b9fdb72c09cd1137a913048f67f34c8161945"} Jul 08 17:34:59.511600 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:59.511547 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sf5mq" podStartSLOduration=66.480962586 podStartE2EDuration="1m11.511532478s" totalImagesPullingTime="5.030569892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:59.51059544 +0000 UTC m=+72.010375185" watchObservedRunningTime="2026-07-08 17:34:59.511532478 +0000 UTC m=+72.011312231" Jul 08 17:34:59.527232 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:34:59.527175 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sgg9w" podStartSLOduration=66.495967586 podStartE2EDuration="1m11.527160146s" totalImagesPullingTime="5.03119256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:59.526293709 +0000 UTC m=+72.026073444" watchObservedRunningTime="2026-07-08 17:34:59.527160146 +0000 UTC m=+72.026939868" Jul 08 17:35:11.688582 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:35:11.688452 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:35:11.688582 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:35:11.688527 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:35:30.501452 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:35:30.501422 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sf5mq" Jul 08 17:35:31.693036 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:35:31.693003 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:35:31.696792 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:35:31.696765 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-cc46778b7-j2cw4" Jul 08 17:35:36.340998 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:35:36.340960 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zdwng_dc5f6ad9-9612-4a84-a375-36b8f3916d69/serve-healthcheck-canary/0.log" Jul 08 17:38:47.899434 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:38:47.899402 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:38:47.904231 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:38:47.904206 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:38:47.906631 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:38:47.906612 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 17:39:52.245356 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.245315 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7df7f97cdf-x55bh"] Jul 08 17:39:52.247910 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.247889 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.250169 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.250151 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 17:39:52.250272 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.250166 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 17:39:52.250380 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.250358 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9nn5h\"" Jul 08 17:39:52.251046 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.251018 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 17:39:52.251279 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.251253 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 17:39:52.253169 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.252678 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-40p9ip1bonshs\"" Jul 08 17:39:52.253169 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.253004 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 17:39:52.260435 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.260413 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7df7f97cdf-x55bh"] Jul 08 17:39:52.315307 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315307 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315309 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-grpc-tls\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315486 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315486 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315420 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315486 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315466 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-metrics-client-ca\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315582 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315495 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7766x\" (UniqueName: \"kubernetes.io/projected/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-kube-api-access-7766x\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315582 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.315582 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.315546 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-tls\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416466 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416420 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-metrics-client-ca\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416471 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7766x\" (UniqueName: \"kubernetes.io/projected/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-kube-api-access-7766x\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-tls\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-grpc-tls\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416578 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.416602 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.416600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.417504 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.417475 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-metrics-client-ca\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.419362 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.419334 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.419461 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.419343 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.419995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.419969 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-tls\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.420084 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.420050 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.420137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.420107 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.420280 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.420258 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-secret-grpc-tls\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.424001 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.423980 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7766x\" (UniqueName: \"kubernetes.io/projected/2b56df8b-1785-4365-9e5e-97f59fa3a4cf-kube-api-access-7766x\") pod \"thanos-querier-7df7f97cdf-x55bh\" (UID: \"2b56df8b-1785-4365-9e5e-97f59fa3a4cf\") " pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.558950 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.558872 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:52.691972 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.691927 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7df7f97cdf-x55bh"] Jul 08 17:39:52.695092 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:39:52.695061 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b56df8b_1785_4365_9e5e_97f59fa3a4cf.slice/crio-61aee9d7f31be27838010932eebeb8d89a44b99e76d962fcf72d9f986adb60cc WatchSource:0}: Error finding container 61aee9d7f31be27838010932eebeb8d89a44b99e76d962fcf72d9f986adb60cc: Status 404 returned error can't find the container with id 61aee9d7f31be27838010932eebeb8d89a44b99e76d962fcf72d9f986adb60cc Jul 08 17:39:52.697142 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:52.697127 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:39:53.452468 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:53.452437 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"61aee9d7f31be27838010932eebeb8d89a44b99e76d962fcf72d9f986adb60cc"} Jul 08 17:39:55.069178 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.069139 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74fd9444fc-lmxzd"] Jul 08 17:39:55.071561 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.071540 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.074254 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.074232 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 17:39:55.074887 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.074867 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 17:39:55.074977 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.074909 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 17:39:55.075031 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.074971 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 17:39:55.075031 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.074979 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gwvj9\"" Jul 08 17:39:55.075522 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.075500 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 17:39:55.081964 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.081946 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 17:39:55.087330 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.087310 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74fd9444fc-lmxzd"] Jul 08 17:39:55.138989 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.138961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139125 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.138996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-federate-client-tls\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139125 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.139024 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-metrics-client-ca\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139205 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.139121 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-secret-telemeter-client\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139205 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.139164 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-serving-certs-ca-bundle\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139205 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.139199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-telemeter-client-tls\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139297 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.139235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4f8w\" (UniqueName: \"kubernetes.io/projected/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-kube-api-access-q4f8w\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.139297 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.139254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240154 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240124 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4f8w\" (UniqueName: \"kubernetes.io/projected/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-kube-api-access-q4f8w\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240302 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240376 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240376 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240355 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-federate-client-tls\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240487 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240396 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-metrics-client-ca\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240487 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240447 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-secret-telemeter-client\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240591 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-serving-certs-ca-bundle\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.240591 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.240532 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-telemeter-client-tls\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.241242 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.241162 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-metrics-client-ca\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.241350 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.241278 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-serving-certs-ca-bundle\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.241503 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.241455 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.243303 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.243281 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.243413 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.243286 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-federate-client-tls\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.243460 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.243418 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-secret-telemeter-client\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.243795 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.243776 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-telemeter-client-tls\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.248949 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.248929 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4f8w\" (UniqueName: \"kubernetes.io/projected/db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44-kube-api-access-q4f8w\") pod \"telemeter-client-74fd9444fc-lmxzd\" (UID: \"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44\") " pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.381719 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.381669 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" Jul 08 17:39:55.850339 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:55.850311 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74fd9444fc-lmxzd"] Jul 08 17:39:55.853262 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:39:55.853231 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb20c1c4_ef59_4ff2_b9fb_1f2c0ced9d44.slice/crio-785aa26fc38973f4819493412df5c3c69144d40460e2fbf1738fa83f382546c5 WatchSource:0}: Error finding container 785aa26fc38973f4819493412df5c3c69144d40460e2fbf1738fa83f382546c5: Status 404 returned error can't find the container with id 785aa26fc38973f4819493412df5c3c69144d40460e2fbf1738fa83f382546c5 Jul 08 17:39:56.463999 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:56.463959 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" event={"ID":"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44","Type":"ContainerStarted","Data":"785aa26fc38973f4819493412df5c3c69144d40460e2fbf1738fa83f382546c5"} Jul 08 17:39:56.465756 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:56.465731 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"40f4d1c6faa05c2bcb470a9560ed49dbc6151076a1feb3bcc300cccf014245d4"} Jul 08 17:39:56.465890 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:56.465786 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"148e5bfc0202f6bcce2e592b64610ac5005be2a38016a03d9a1d88bc1bf2dd96"} Jul 08 17:39:56.465890 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:56.465811 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"f718c5bd96162864dc38fc7e20ca03ee6e31b561c4042abc2a9d7e54a765423b"} Jul 08 17:39:57.472108 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:57.472072 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"8748fe86718aa24e85d1eb52e7c11a781d74a8f41653f0ce51c94979f62b50ca"} Jul 08 17:39:57.472484 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:57.472134 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"eb18b9175d7a847e7647face178f405475c143f5567ec11133bfa5fc6716cde1"} Jul 08 17:39:57.472484 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:57.472159 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" event={"ID":"2b56df8b-1785-4365-9e5e-97f59fa3a4cf","Type":"ContainerStarted","Data":"2e537706d79afc2834f54f6c94c6d8e15e1d66a2a6e3906dbfd84bd9cc389857"} Jul 08 17:39:57.472484 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:57.472230 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:39:57.498730 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:57.498670 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" podStartSLOduration=1.499320395 podStartE2EDuration="5.498650307s" totalImagesPullingTime="3.999329912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:39:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:39:57.497128069 +0000 UTC m=+369.996907850" watchObservedRunningTime="2026-07-08 17:39:57.498650307 +0000 UTC m=+369.998430031" Jul 08 17:39:59.481460 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:39:59.481425 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" event={"ID":"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44","Type":"ContainerStarted","Data":"0bb47f1f6671fa627220a51cc23164727e7494722d186ba6953e21825c8a0dfd"} Jul 08 17:40:00.486987 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:40:00.486945 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" event={"ID":"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44","Type":"ContainerStarted","Data":"e17c8e732951ee8707d6de006a20ec6f56800e265b1b4ba10bd35e246dc15555"} Jul 08 17:40:00.487400 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:40:00.487029 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" event={"ID":"db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44","Type":"ContainerStarted","Data":"0ad180388bb4b36c8dba829830d190b3fade57ac2ad8df710df4f8b931783db1"} Jul 08 17:40:00.509835 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:40:00.509776 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74fd9444fc-lmxzd" podStartSLOduration=1.762933768 podStartE2EDuration="5.509756238s" totalImagesPullingTime="3.74682247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:39:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:40:00.508720226 +0000 UTC m=+373.008499970" watchObservedRunningTime="2026-07-08 17:40:00.509756238 +0000 UTC m=+373.009535962" Jul 08 17:40:03.483289 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:40:03.483257 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7df7f97cdf-x55bh" Jul 08 17:42:05.076371 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.076337 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf"] Jul 08 17:42:05.079543 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.079526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.084151 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.084129 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8v26j\"" Jul 08 17:42:05.084297 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.084281 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 17:42:05.090964 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.090947 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 17:42:05.115392 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.115354 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf"] Jul 08 17:42:05.121261 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.121233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.121368 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.121280 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fctr2\" (UniqueName: \"kubernetes.io/projected/da65a172-24e6-472d-a151-f13b20044aef-kube-api-access-fctr2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.121368 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.121346 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.221783 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.221742 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.221976 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.221798 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fctr2\" (UniqueName: \"kubernetes.io/projected/da65a172-24e6-472d-a151-f13b20044aef-kube-api-access-fctr2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.221976 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.221871 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.222234 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.222214 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.222271 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.222224 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.231042 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.231008 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fctr2\" (UniqueName: \"kubernetes.io/projected/da65a172-24e6-472d-a151-f13b20044aef-kube-api-access-fctr2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.388644 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.388558 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:05.520211 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.520186 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf"] Jul 08 17:42:05.523009 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:05.522975 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda65a172_24e6_472d_a151_f13b20044aef.slice/crio-1390b32e56036abbb4906881315a054389c6d77a4303d7b14ffd3c638064fb15 WatchSource:0}: Error finding container 1390b32e56036abbb4906881315a054389c6d77a4303d7b14ffd3c638064fb15: Status 404 returned error can't find the container with id 1390b32e56036abbb4906881315a054389c6d77a4303d7b14ffd3c638064fb15 Jul 08 17:42:05.929924 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:05.929892 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" event={"ID":"da65a172-24e6-472d-a151-f13b20044aef","Type":"ContainerStarted","Data":"1390b32e56036abbb4906881315a054389c6d77a4303d7b14ffd3c638064fb15"} Jul 08 17:42:13.963291 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:13.963252 2574 generic.go:356] "Generic (PLEG): container finished" podID="da65a172-24e6-472d-a151-f13b20044aef" containerID="6aa2e736e696cd23448c06eb0e159276d2f8401162b3887612ebc12669d86352" exitCode=0 Jul 08 17:42:13.963688 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:13.963308 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" event={"ID":"da65a172-24e6-472d-a151-f13b20044aef","Type":"ContainerDied","Data":"6aa2e736e696cd23448c06eb0e159276d2f8401162b3887612ebc12669d86352"} Jul 08 17:42:16.977186 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:16.977144 2574 generic.go:356] "Generic (PLEG): container finished" podID="da65a172-24e6-472d-a151-f13b20044aef" containerID="eba0c60d14eb03fbbb9b322e3a23c6c64a1492d8a0880a2b24de636f7a100dc5" exitCode=0 Jul 08 17:42:16.977568 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:16.977230 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" event={"ID":"da65a172-24e6-472d-a151-f13b20044aef","Type":"ContainerDied","Data":"eba0c60d14eb03fbbb9b322e3a23c6c64a1492d8a0880a2b24de636f7a100dc5"} Jul 08 17:42:26.012163 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:26.012127 2574 generic.go:356] "Generic (PLEG): container finished" podID="da65a172-24e6-472d-a151-f13b20044aef" containerID="b1bbf1495ea089c508dbae124e1a82a4d51b6fea35b608b2cc40dd5055ed69bf" exitCode=0 Jul 08 17:42:26.012524 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:26.012190 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" event={"ID":"da65a172-24e6-472d-a151-f13b20044aef","Type":"ContainerDied","Data":"b1bbf1495ea089c508dbae124e1a82a4d51b6fea35b608b2cc40dd5055ed69bf"} Jul 08 17:42:27.169213 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.169187 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:27.276414 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.276336 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fctr2\" (UniqueName: \"kubernetes.io/projected/da65a172-24e6-472d-a151-f13b20044aef-kube-api-access-fctr2\") pod \"da65a172-24e6-472d-a151-f13b20044aef\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " Jul 08 17:42:27.276414 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.276375 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-util\") pod \"da65a172-24e6-472d-a151-f13b20044aef\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " Jul 08 17:42:27.276414 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.276404 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-bundle\") pod \"da65a172-24e6-472d-a151-f13b20044aef\" (UID: \"da65a172-24e6-472d-a151-f13b20044aef\") " Jul 08 17:42:27.277093 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.277067 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-bundle" (OuterVolumeSpecName: "bundle") pod "da65a172-24e6-472d-a151-f13b20044aef" (UID: "da65a172-24e6-472d-a151-f13b20044aef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:42:27.278715 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.278694 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da65a172-24e6-472d-a151-f13b20044aef-kube-api-access-fctr2" (OuterVolumeSpecName: "kube-api-access-fctr2") pod "da65a172-24e6-472d-a151-f13b20044aef" (UID: "da65a172-24e6-472d-a151-f13b20044aef"). InnerVolumeSpecName "kube-api-access-fctr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:42:27.280748 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.280727 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-util" (OuterVolumeSpecName: "util") pod "da65a172-24e6-472d-a151-f13b20044aef" (UID: "da65a172-24e6-472d-a151-f13b20044aef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:42:27.377930 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.377897 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fctr2\" (UniqueName: \"kubernetes.io/projected/da65a172-24e6-472d-a151-f13b20044aef-kube-api-access-fctr2\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:42:27.377930 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.377923 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:42:27.377930 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:27.377932 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da65a172-24e6-472d-a151-f13b20044aef-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:42:28.024669 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:28.024639 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" Jul 08 17:42:28.024842 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:28.024641 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540txbrf" event={"ID":"da65a172-24e6-472d-a151-f13b20044aef","Type":"ContainerDied","Data":"1390b32e56036abbb4906881315a054389c6d77a4303d7b14ffd3c638064fb15"} Jul 08 17:42:28.024842 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:28.024758 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1390b32e56036abbb4906881315a054389c6d77a4303d7b14ffd3c638064fb15" Jul 08 17:42:33.365652 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365615 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v"] Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365905 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="extract" Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365916 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="extract" Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365930 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="pull" Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365945 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="pull" Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365961 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="util" Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.365966 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="util" Jul 08 17:42:33.366269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.366011 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="da65a172-24e6-472d-a151-f13b20044aef" containerName="extract" Jul 08 17:42:33.369386 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.369367 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.377256 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.377236 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 17:42:33.377355 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.377328 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:42:33.377479 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.377462 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cj7hf\"" Jul 08 17:42:33.391108 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.391082 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v"] Jul 08 17:42:33.424346 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.424320 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffd46e85-e4be-47ce-b3a5-8884cd2f8556-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ckb5v\" (UID: \"ffd46e85-e4be-47ce-b3a5-8884cd2f8556\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.424440 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.424363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5jmr\" (UniqueName: \"kubernetes.io/projected/ffd46e85-e4be-47ce-b3a5-8884cd2f8556-kube-api-access-x5jmr\") pod \"cert-manager-operator-controller-manager-55c444bf86-ckb5v\" (UID: \"ffd46e85-e4be-47ce-b3a5-8884cd2f8556\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.525486 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.525431 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5jmr\" (UniqueName: \"kubernetes.io/projected/ffd46e85-e4be-47ce-b3a5-8884cd2f8556-kube-api-access-x5jmr\") pod \"cert-manager-operator-controller-manager-55c444bf86-ckb5v\" (UID: \"ffd46e85-e4be-47ce-b3a5-8884cd2f8556\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.525694 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.525546 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffd46e85-e4be-47ce-b3a5-8884cd2f8556-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ckb5v\" (UID: \"ffd46e85-e4be-47ce-b3a5-8884cd2f8556\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.526018 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.525994 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffd46e85-e4be-47ce-b3a5-8884cd2f8556-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ckb5v\" (UID: \"ffd46e85-e4be-47ce-b3a5-8884cd2f8556\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.538657 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.538634 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5jmr\" (UniqueName: \"kubernetes.io/projected/ffd46e85-e4be-47ce-b3a5-8884cd2f8556-kube-api-access-x5jmr\") pod \"cert-manager-operator-controller-manager-55c444bf86-ckb5v\" (UID: \"ffd46e85-e4be-47ce-b3a5-8884cd2f8556\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.678360 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.678269 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" Jul 08 17:42:33.809787 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:33.809753 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v"] Jul 08 17:42:33.813296 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:33.813271 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd46e85_e4be_47ce_b3a5_8884cd2f8556.slice/crio-6991c586f53d00c13ab361f07bbdc977462b2c95edf1c60bbe353b2691a55990 WatchSource:0}: Error finding container 6991c586f53d00c13ab361f07bbdc977462b2c95edf1c60bbe353b2691a55990: Status 404 returned error can't find the container with id 6991c586f53d00c13ab361f07bbdc977462b2c95edf1c60bbe353b2691a55990 Jul 08 17:42:34.048109 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:34.048024 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" event={"ID":"ffd46e85-e4be-47ce-b3a5-8884cd2f8556","Type":"ContainerStarted","Data":"6991c586f53d00c13ab361f07bbdc977462b2c95edf1c60bbe353b2691a55990"} Jul 08 17:42:36.061947 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.061866 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" event={"ID":"ffd46e85-e4be-47ce-b3a5-8884cd2f8556","Type":"ContainerStarted","Data":"bfb133dd1e0de5b998f64b26357e354abf3b88307b5e4f7cce7891ee844c3707"} Jul 08 17:42:36.080777 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.080715 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ckb5v" podStartSLOduration=1.317824768 podStartE2EDuration="3.080699179s" totalImagesPullingTime="1.762874411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:42:36.078427443 +0000 UTC m=+528.578207167" watchObservedRunningTime="2026-07-08 17:42:36.080699179 +0000 UTC m=+528.580478901" Jul 08 17:42:36.775213 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.775177 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7spj"] Jul 08 17:42:36.778200 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.778177 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:36.781262 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.781239 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 17:42:36.781501 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.781480 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 17:42:36.792630 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.792606 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7spj"] Jul 08 17:42:36.853955 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.853920 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg46z\" (UniqueName: \"kubernetes.io/projected/0a9e8c99-8da6-42d0-8a1b-d96da93508af-kube-api-access-sg46z\") pod \"cert-manager-webhook-6486ffd997-r7spj\" (UID: \"0a9e8c99-8da6-42d0-8a1b-d96da93508af\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:36.854124 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.853984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a9e8c99-8da6-42d0-8a1b-d96da93508af-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7spj\" (UID: \"0a9e8c99-8da6-42d0-8a1b-d96da93508af\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:36.954909 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.954863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a9e8c99-8da6-42d0-8a1b-d96da93508af-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7spj\" (UID: \"0a9e8c99-8da6-42d0-8a1b-d96da93508af\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:36.955084 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.954947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg46z\" (UniqueName: \"kubernetes.io/projected/0a9e8c99-8da6-42d0-8a1b-d96da93508af-kube-api-access-sg46z\") pod \"cert-manager-webhook-6486ffd997-r7spj\" (UID: \"0a9e8c99-8da6-42d0-8a1b-d96da93508af\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:36.966624 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.966586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a9e8c99-8da6-42d0-8a1b-d96da93508af-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7spj\" (UID: \"0a9e8c99-8da6-42d0-8a1b-d96da93508af\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:36.966741 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:36.966657 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg46z\" (UniqueName: \"kubernetes.io/projected/0a9e8c99-8da6-42d0-8a1b-d96da93508af-kube-api-access-sg46z\") pod \"cert-manager-webhook-6486ffd997-r7spj\" (UID: \"0a9e8c99-8da6-42d0-8a1b-d96da93508af\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:37.098810 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.098774 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:37.227166 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.227075 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7spj"] Jul 08 17:42:37.229711 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:37.229683 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a9e8c99_8da6_42d0_8a1b_d96da93508af.slice/crio-095803ffa4908d80d76e0d56602f1e78dff7cb3152f10b5b5e5b29e6c9420ea3 WatchSource:0}: Error finding container 095803ffa4908d80d76e0d56602f1e78dff7cb3152f10b5b5e5b29e6c9420ea3: Status 404 returned error can't find the container with id 095803ffa4908d80d76e0d56602f1e78dff7cb3152f10b5b5e5b29e6c9420ea3 Jul 08 17:42:37.395359 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.395275 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs"] Jul 08 17:42:37.398508 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.398489 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.400772 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.400751 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 17:42:37.400896 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.400877 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8v26j\"" Jul 08 17:42:37.400942 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.400921 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 17:42:37.408694 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.408665 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs"] Jul 08 17:42:37.459869 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.459808 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.460059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.459879 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt777\" (UniqueName: \"kubernetes.io/projected/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-kube-api-access-jt777\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.460059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.459989 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.560864 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.560810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.561043 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.560880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.561043 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.560907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt777\" (UniqueName: \"kubernetes.io/projected/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-kube-api-access-jt777\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.561251 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.561231 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.561320 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.561260 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.594358 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.594330 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt777\" (UniqueName: \"kubernetes.io/projected/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-kube-api-access-jt777\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.708091 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.708006 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:37.836050 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:37.836018 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs"] Jul 08 17:42:37.838939 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:37.838907 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25fdc2b_ac41_4b5d_aceb_c166f892f5e2.slice/crio-efe11826692547d97750f617de1dc5319dc1773056d60f312f64202cc3a11c3a WatchSource:0}: Error finding container efe11826692547d97750f617de1dc5319dc1773056d60f312f64202cc3a11c3a: Status 404 returned error can't find the container with id efe11826692547d97750f617de1dc5319dc1773056d60f312f64202cc3a11c3a Jul 08 17:42:38.066367 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:38.066281 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" event={"ID":"0a9e8c99-8da6-42d0-8a1b-d96da93508af","Type":"ContainerStarted","Data":"095803ffa4908d80d76e0d56602f1e78dff7cb3152f10b5b5e5b29e6c9420ea3"} Jul 08 17:42:38.067684 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:38.067659 2574 generic.go:356] "Generic (PLEG): container finished" podID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerID="2005e841610df8d5970313e0df6fc85cc263d4aa1e642ab14b768bc94ce4afbd" exitCode=0 Jul 08 17:42:38.067811 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:38.067742 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" event={"ID":"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2","Type":"ContainerDied","Data":"2005e841610df8d5970313e0df6fc85cc263d4aa1e642ab14b768bc94ce4afbd"} Jul 08 17:42:38.067811 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:38.067794 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" event={"ID":"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2","Type":"ContainerStarted","Data":"efe11826692547d97750f617de1dc5319dc1773056d60f312f64202cc3a11c3a"} Jul 08 17:42:40.295448 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.295415 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9d86z"] Jul 08 17:42:40.298875 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.298853 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.301134 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.301116 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sx4rt\"" Jul 08 17:42:40.308965 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.308938 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9d86z"] Jul 08 17:42:40.384632 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.384590 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0a01b96-7bca-4d44-82c0-767b87bc799f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9d86z\" (UID: \"f0a01b96-7bca-4d44-82c0-767b87bc799f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.384802 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.384660 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbmt\" (UniqueName: \"kubernetes.io/projected/f0a01b96-7bca-4d44-82c0-767b87bc799f-kube-api-access-4jbmt\") pod \"cert-manager-cainjector-559cdd8c87-9d86z\" (UID: \"f0a01b96-7bca-4d44-82c0-767b87bc799f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.485917 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.485880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0a01b96-7bca-4d44-82c0-767b87bc799f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9d86z\" (UID: \"f0a01b96-7bca-4d44-82c0-767b87bc799f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.486089 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.485949 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbmt\" (UniqueName: \"kubernetes.io/projected/f0a01b96-7bca-4d44-82c0-767b87bc799f-kube-api-access-4jbmt\") pod \"cert-manager-cainjector-559cdd8c87-9d86z\" (UID: \"f0a01b96-7bca-4d44-82c0-767b87bc799f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.495844 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.495792 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0a01b96-7bca-4d44-82c0-767b87bc799f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9d86z\" (UID: \"f0a01b96-7bca-4d44-82c0-767b87bc799f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.496049 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.496029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbmt\" (UniqueName: \"kubernetes.io/projected/f0a01b96-7bca-4d44-82c0-767b87bc799f-kube-api-access-4jbmt\") pod \"cert-manager-cainjector-559cdd8c87-9d86z\" (UID: \"f0a01b96-7bca-4d44-82c0-767b87bc799f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:40.610120 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:40.610077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" Jul 08 17:42:41.085996 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:41.085736 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" event={"ID":"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2","Type":"ContainerStarted","Data":"e90a6352737e55b68d41497d71792f72fa99e02ec1a6caba8b36fdefb4978ddb"} Jul 08 17:42:41.088385 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:41.088331 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" event={"ID":"0a9e8c99-8da6-42d0-8a1b-d96da93508af","Type":"ContainerStarted","Data":"2d7368663c049fb970201311287dcab7649f193696d5a0807e18a2481b4cf902"} Jul 08 17:42:41.088615 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:41.088583 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:41.128453 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:41.128037 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9d86z"] Jul 08 17:42:41.149587 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:41.149560 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0a01b96_7bca_4d44_82c0_767b87bc799f.slice/crio-39b12f7f42658cb5e55a8a3e856be221138254f77c6ece2cf9a4fcbfae3ba213 WatchSource:0}: Error finding container 39b12f7f42658cb5e55a8a3e856be221138254f77c6ece2cf9a4fcbfae3ba213: Status 404 returned error can't find the container with id 39b12f7f42658cb5e55a8a3e856be221138254f77c6ece2cf9a4fcbfae3ba213 Jul 08 17:42:41.152081 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:41.151972 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" podStartSLOduration=1.393922468 podStartE2EDuration="5.151952164s" totalImagesPullingTime="3.758029696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:42:41.151427973 +0000 UTC m=+533.651207760" watchObservedRunningTime="2026-07-08 17:42:41.151952164 +0000 UTC m=+533.651731888" Jul 08 17:42:42.093699 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:42.093665 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" event={"ID":"f0a01b96-7bca-4d44-82c0-767b87bc799f","Type":"ContainerStarted","Data":"0df450949ffa6942e094d8ccc3ce446760af3d80fa01f7766961d322aa5a3acc"} Jul 08 17:42:42.094143 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:42.093706 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" event={"ID":"f0a01b96-7bca-4d44-82c0-767b87bc799f","Type":"ContainerStarted","Data":"39b12f7f42658cb5e55a8a3e856be221138254f77c6ece2cf9a4fcbfae3ba213"} Jul 08 17:42:42.095488 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:42.095465 2574 generic.go:356] "Generic (PLEG): container finished" podID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerID="e90a6352737e55b68d41497d71792f72fa99e02ec1a6caba8b36fdefb4978ddb" exitCode=0 Jul 08 17:42:42.095614 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:42.095545 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" event={"ID":"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2","Type":"ContainerDied","Data":"e90a6352737e55b68d41497d71792f72fa99e02ec1a6caba8b36fdefb4978ddb"} Jul 08 17:42:42.126384 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:42.126338 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9d86z" podStartSLOduration=2.126324532 podStartE2EDuration="2.126324532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:42:42.109716295 +0000 UTC m=+534.609496019" watchObservedRunningTime="2026-07-08 17:42:42.126324532 +0000 UTC m=+534.626104262" Jul 08 17:42:43.101510 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:43.101469 2574 generic.go:356] "Generic (PLEG): container finished" podID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerID="c27a2eb6c58f3824019ff6e9df15d64630748495701245baecada30677da69d7" exitCode=0 Jul 08 17:42:43.101928 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:43.101553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" event={"ID":"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2","Type":"ContainerDied","Data":"c27a2eb6c58f3824019ff6e9df15d64630748495701245baecada30677da69d7"} Jul 08 17:42:44.227802 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.227778 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:44.319320 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.319292 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt777\" (UniqueName: \"kubernetes.io/projected/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-kube-api-access-jt777\") pod \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " Jul 08 17:42:44.319472 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.319349 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-bundle\") pod \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " Jul 08 17:42:44.319472 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.319428 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-util\") pod \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\" (UID: \"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2\") " Jul 08 17:42:44.319705 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.319682 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-bundle" (OuterVolumeSpecName: "bundle") pod "c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" (UID: "c25fdc2b-ac41-4b5d-aceb-c166f892f5e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:42:44.321549 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.321518 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-kube-api-access-jt777" (OuterVolumeSpecName: "kube-api-access-jt777") pod "c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" (UID: "c25fdc2b-ac41-4b5d-aceb-c166f892f5e2"). InnerVolumeSpecName "kube-api-access-jt777". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:42:44.324245 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.324225 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-util" (OuterVolumeSpecName: "util") pod "c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" (UID: "c25fdc2b-ac41-4b5d-aceb-c166f892f5e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:42:44.420274 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.420202 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:42:44.420274 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.420229 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt777\" (UniqueName: \"kubernetes.io/projected/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-kube-api-access-jt777\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:42:44.420274 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:44.420239 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25fdc2b-ac41-4b5d-aceb-c166f892f5e2-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:42:45.111797 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:45.111758 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" event={"ID":"c25fdc2b-ac41-4b5d-aceb-c166f892f5e2","Type":"ContainerDied","Data":"efe11826692547d97750f617de1dc5319dc1773056d60f312f64202cc3a11c3a"} Jul 08 17:42:45.111797 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:45.111796 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flk5bs" Jul 08 17:42:45.112005 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:45.111806 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efe11826692547d97750f617de1dc5319dc1773056d60f312f64202cc3a11c3a" Jul 08 17:42:47.097592 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:47.097559 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r7spj" Jul 08 17:42:49.300373 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300338 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w5cdh"] Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300603 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="pull" Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300613 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="pull" Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300622 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="util" Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300627 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="util" Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300644 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="extract" Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300650 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="extract" Jul 08 17:42:49.300731 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.300701 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c25fdc2b-ac41-4b5d-aceb-c166f892f5e2" containerName="extract" Jul 08 17:42:49.303375 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.303356 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.305517 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.305500 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-d84sg\"" Jul 08 17:42:49.313689 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.313664 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w5cdh"] Jul 08 17:42:49.461629 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.461598 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5jrz\" (UniqueName: \"kubernetes.io/projected/41ad12f6-13d6-4f9a-899d-a9b9184dd720-kube-api-access-t5jrz\") pod \"cert-manager-55b4d954d5-w5cdh\" (UID: \"41ad12f6-13d6-4f9a-899d-a9b9184dd720\") " pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.461818 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.461646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41ad12f6-13d6-4f9a-899d-a9b9184dd720-bound-sa-token\") pod \"cert-manager-55b4d954d5-w5cdh\" (UID: \"41ad12f6-13d6-4f9a-899d-a9b9184dd720\") " pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.563040 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.562955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5jrz\" (UniqueName: \"kubernetes.io/projected/41ad12f6-13d6-4f9a-899d-a9b9184dd720-kube-api-access-t5jrz\") pod \"cert-manager-55b4d954d5-w5cdh\" (UID: \"41ad12f6-13d6-4f9a-899d-a9b9184dd720\") " pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.563040 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.563024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41ad12f6-13d6-4f9a-899d-a9b9184dd720-bound-sa-token\") pod \"cert-manager-55b4d954d5-w5cdh\" (UID: \"41ad12f6-13d6-4f9a-899d-a9b9184dd720\") " pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.571894 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.571865 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41ad12f6-13d6-4f9a-899d-a9b9184dd720-bound-sa-token\") pod \"cert-manager-55b4d954d5-w5cdh\" (UID: \"41ad12f6-13d6-4f9a-899d-a9b9184dd720\") " pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.572092 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.572071 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5jrz\" (UniqueName: \"kubernetes.io/projected/41ad12f6-13d6-4f9a-899d-a9b9184dd720-kube-api-access-t5jrz\") pod \"cert-manager-55b4d954d5-w5cdh\" (UID: \"41ad12f6-13d6-4f9a-899d-a9b9184dd720\") " pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.612889 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.612857 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-w5cdh" Jul 08 17:42:49.753550 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:49.753518 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-w5cdh"] Jul 08 17:42:49.756492 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:49.756465 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41ad12f6_13d6_4f9a_899d_a9b9184dd720.slice/crio-9d76567da71a5a43d5fb1044a2fe2082fc11d1401229534aee4bfcf213c32216 WatchSource:0}: Error finding container 9d76567da71a5a43d5fb1044a2fe2082fc11d1401229534aee4bfcf213c32216: Status 404 returned error can't find the container with id 9d76567da71a5a43d5fb1044a2fe2082fc11d1401229534aee4bfcf213c32216 Jul 08 17:42:50.134354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:50.134315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w5cdh" event={"ID":"41ad12f6-13d6-4f9a-899d-a9b9184dd720","Type":"ContainerStarted","Data":"6391f2165430449eb2c1a6c04e3075ab019c3f72917e3ef72f0ab0c78cd5feaa"} Jul 08 17:42:50.134354 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:50.134359 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-w5cdh" event={"ID":"41ad12f6-13d6-4f9a-899d-a9b9184dd720","Type":"ContainerStarted","Data":"9d76567da71a5a43d5fb1044a2fe2082fc11d1401229534aee4bfcf213c32216"} Jul 08 17:42:50.153094 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:50.153032 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-w5cdh" podStartSLOduration=1.1530126219999999 podStartE2EDuration="1.153012622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:42:50.152436412 +0000 UTC m=+542.652216146" watchObservedRunningTime="2026-07-08 17:42:50.153012622 +0000 UTC m=+542.652792367" Jul 08 17:42:57.565294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.565258 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5"] Jul 08 17:42:57.570101 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.570079 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.572456 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.572421 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 17:42:57.572456 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.572421 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 17:42:57.573176 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.573154 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8v26j\"" Jul 08 17:42:57.578738 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.578718 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5"] Jul 08 17:42:57.732936 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.732888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.733111 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.732959 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.733111 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.732998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5z9v\" (UniqueName: \"kubernetes.io/projected/0cb478f1-c35a-4bef-8bce-330893602202-kube-api-access-g5z9v\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.833969 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.833934 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.834127 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.833983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.834127 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.834025 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5z9v\" (UniqueName: \"kubernetes.io/projected/0cb478f1-c35a-4bef-8bce-330893602202-kube-api-access-g5z9v\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.834417 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.834395 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.834456 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.834402 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.842582 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.842559 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5z9v\" (UniqueName: \"kubernetes.io/projected/0cb478f1-c35a-4bef-8bce-330893602202-kube-api-access-g5z9v\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:57.880849 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:57.880799 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:42:58.013928 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:58.013865 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5"] Jul 08 17:42:58.016538 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:42:58.016508 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb478f1_c35a_4bef_8bce_330893602202.slice/crio-15b2b36ae978355ee12eac2d555688ae494ad7240d86cf71bbd95a77296d5a4e WatchSource:0}: Error finding container 15b2b36ae978355ee12eac2d555688ae494ad7240d86cf71bbd95a77296d5a4e: Status 404 returned error can't find the container with id 15b2b36ae978355ee12eac2d555688ae494ad7240d86cf71bbd95a77296d5a4e Jul 08 17:42:58.169652 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:58.169619 2574 generic.go:356] "Generic (PLEG): container finished" podID="0cb478f1-c35a-4bef-8bce-330893602202" containerID="266f5dca120606eb76a5fac353fd275c734231ca7f6edf04977e27897f137bee" exitCode=0 Jul 08 17:42:58.169818 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:58.169688 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" event={"ID":"0cb478f1-c35a-4bef-8bce-330893602202","Type":"ContainerDied","Data":"266f5dca120606eb76a5fac353fd275c734231ca7f6edf04977e27897f137bee"} Jul 08 17:42:58.169818 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:58.169729 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" event={"ID":"0cb478f1-c35a-4bef-8bce-330893602202","Type":"ContainerStarted","Data":"15b2b36ae978355ee12eac2d555688ae494ad7240d86cf71bbd95a77296d5a4e"} Jul 08 17:42:59.175885 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:59.175787 2574 generic.go:356] "Generic (PLEG): container finished" podID="0cb478f1-c35a-4bef-8bce-330893602202" containerID="229f0e0dcdf11031686378eba62ef96ab1fdabd5378d5d2eeca57f68ef9a18e2" exitCode=0 Jul 08 17:42:59.175885 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:42:59.175843 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" event={"ID":"0cb478f1-c35a-4bef-8bce-330893602202","Type":"ContainerDied","Data":"229f0e0dcdf11031686378eba62ef96ab1fdabd5378d5d2eeca57f68ef9a18e2"} Jul 08 17:43:00.181994 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:00.181959 2574 generic.go:356] "Generic (PLEG): container finished" podID="0cb478f1-c35a-4bef-8bce-330893602202" containerID="bc85c6dd13ff6bfcbf8ca0a38ff0c013bec6fbd3dd3d1b8797093f31389da51f" exitCode=0 Jul 08 17:43:00.182373 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:00.182028 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" event={"ID":"0cb478f1-c35a-4bef-8bce-330893602202","Type":"ContainerDied","Data":"bc85c6dd13ff6bfcbf8ca0a38ff0c013bec6fbd3dd3d1b8797093f31389da51f"} Jul 08 17:43:01.314506 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.314478 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:43:01.464916 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.464811 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-util\") pod \"0cb478f1-c35a-4bef-8bce-330893602202\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " Jul 08 17:43:01.464916 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.464879 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-bundle\") pod \"0cb478f1-c35a-4bef-8bce-330893602202\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " Jul 08 17:43:01.465092 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.464924 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5z9v\" (UniqueName: \"kubernetes.io/projected/0cb478f1-c35a-4bef-8bce-330893602202-kube-api-access-g5z9v\") pod \"0cb478f1-c35a-4bef-8bce-330893602202\" (UID: \"0cb478f1-c35a-4bef-8bce-330893602202\") " Jul 08 17:43:01.465603 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.465571 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-bundle" (OuterVolumeSpecName: "bundle") pod "0cb478f1-c35a-4bef-8bce-330893602202" (UID: "0cb478f1-c35a-4bef-8bce-330893602202"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:43:01.467153 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.467133 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb478f1-c35a-4bef-8bce-330893602202-kube-api-access-g5z9v" (OuterVolumeSpecName: "kube-api-access-g5z9v") pod "0cb478f1-c35a-4bef-8bce-330893602202" (UID: "0cb478f1-c35a-4bef-8bce-330893602202"). InnerVolumeSpecName "kube-api-access-g5z9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:43:01.470527 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.470504 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-util" (OuterVolumeSpecName: "util") pod "0cb478f1-c35a-4bef-8bce-330893602202" (UID: "0cb478f1-c35a-4bef-8bce-330893602202"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:43:01.566335 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.566300 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:43:01.566335 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.566330 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb478f1-c35a-4bef-8bce-330893602202-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:43:01.566335 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:01.566342 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5z9v\" (UniqueName: \"kubernetes.io/projected/0cb478f1-c35a-4bef-8bce-330893602202-kube-api-access-g5z9v\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:43:02.192698 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:02.192665 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" event={"ID":"0cb478f1-c35a-4bef-8bce-330893602202","Type":"ContainerDied","Data":"15b2b36ae978355ee12eac2d555688ae494ad7240d86cf71bbd95a77296d5a4e"} Jul 08 17:43:02.192870 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:02.192716 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15b2b36ae978355ee12eac2d555688ae494ad7240d86cf71bbd95a77296d5a4e" Jul 08 17:43:02.192870 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:02.192671 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6qj5" Jul 08 17:43:12.350535 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350497 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh"] Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350779 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="extract" Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350790 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="extract" Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350806 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="util" Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350811 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="util" Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350818 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="pull" Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350836 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="pull" Jul 08 17:43:12.350995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.350892 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb478f1-c35a-4bef-8bce-330893602202" containerName="extract" Jul 08 17:43:12.352835 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.352811 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.355295 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.355274 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 17:43:12.355295 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.355284 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 17:43:12.355498 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.355390 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 17:43:12.355498 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.355429 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cfbk9\"" Jul 08 17:43:12.355498 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.355435 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 17:43:12.375424 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.375392 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh"] Jul 08 17:43:12.454299 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.454265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q24p7\" (UniqueName: \"kubernetes.io/projected/483dba4c-3621-4126-ae6f-738ee2d29df1-kube-api-access-q24p7\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.454299 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.454301 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/483dba4c-3621-4126-ae6f-738ee2d29df1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.454489 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.454327 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483dba4c-3621-4126-ae6f-738ee2d29df1-tmp\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.454489 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.454343 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/483dba4c-3621-4126-ae6f-738ee2d29df1-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.454489 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.454387 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/483dba4c-3621-4126-ae6f-738ee2d29df1-manifests\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.554983 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.554952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24p7\" (UniqueName: \"kubernetes.io/projected/483dba4c-3621-4126-ae6f-738ee2d29df1-kube-api-access-q24p7\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.554983 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.554988 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/483dba4c-3621-4126-ae6f-738ee2d29df1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.555189 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.555013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483dba4c-3621-4126-ae6f-738ee2d29df1-tmp\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.555189 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.555143 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/483dba4c-3621-4126-ae6f-738ee2d29df1-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.555295 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.555190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/483dba4c-3621-4126-ae6f-738ee2d29df1-manifests\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.555563 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.555532 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/483dba4c-3621-4126-ae6f-738ee2d29df1-manifests\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.557422 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.557400 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/483dba4c-3621-4126-ae6f-738ee2d29df1-tmp\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.557674 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.557651 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/483dba4c-3621-4126-ae6f-738ee2d29df1-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.557725 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.557702 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/483dba4c-3621-4126-ae6f-738ee2d29df1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.571533 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.571512 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q24p7\" (UniqueName: \"kubernetes.io/projected/483dba4c-3621-4126-ae6f-738ee2d29df1-kube-api-access-q24p7\") pod \"opendatahub-operator-controller-manager-5c548d5447-9s8jh\" (UID: \"483dba4c-3621-4126-ae6f-738ee2d29df1\") " pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.662813 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.662741 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:12.798729 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:12.798692 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh"] Jul 08 17:43:12.802195 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:43:12.802163 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483dba4c_3621_4126_ae6f_738ee2d29df1.slice/crio-21051341adc790f5c9c1f8e2ff892eaf72bd1d7280ec8e6aa23a7b41485d4f33 WatchSource:0}: Error finding container 21051341adc790f5c9c1f8e2ff892eaf72bd1d7280ec8e6aa23a7b41485d4f33: Status 404 returned error can't find the container with id 21051341adc790f5c9c1f8e2ff892eaf72bd1d7280ec8e6aa23a7b41485d4f33 Jul 08 17:43:13.239594 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:13.239559 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" event={"ID":"483dba4c-3621-4126-ae6f-738ee2d29df1","Type":"ContainerStarted","Data":"21051341adc790f5c9c1f8e2ff892eaf72bd1d7280ec8e6aa23a7b41485d4f33"} Jul 08 17:43:16.254997 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:16.254911 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" event={"ID":"483dba4c-3621-4126-ae6f-738ee2d29df1","Type":"ContainerStarted","Data":"ba84ee9498052bf423cd436431f397272ed037ba93c008769ff932924b36a1b6"} Jul 08 17:43:17.262397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:17.262286 2574 generic.go:356] "Generic (PLEG): container finished" podID="483dba4c-3621-4126-ae6f-738ee2d29df1" containerID="ba84ee9498052bf423cd436431f397272ed037ba93c008769ff932924b36a1b6" exitCode=0 Jul 08 17:43:17.262397 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:17.262370 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" event={"ID":"483dba4c-3621-4126-ae6f-738ee2d29df1","Type":"ContainerDied","Data":"ba84ee9498052bf423cd436431f397272ed037ba93c008769ff932924b36a1b6"} Jul 08 17:43:18.269029 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:18.268990 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" event={"ID":"483dba4c-3621-4126-ae6f-738ee2d29df1","Type":"ContainerStarted","Data":"978eaa69782740e571beae15f15b6c78e7cbccb81c24e1fb88a1f73be1043c36"} Jul 08 17:43:18.269385 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:18.269126 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:18.292272 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:18.292217 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" podStartSLOduration=1.74905954 podStartE2EDuration="6.292197262s" totalImagesPullingTime="3.510755908s" totalInitContainerRuntime="1.032381814s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:43:18.289888817 +0000 UTC m=+570.789668550" watchObservedRunningTime="2026-07-08 17:43:18.292197262 +0000 UTC m=+570.791976984" Jul 08 17:43:29.275553 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:29.275519 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c548d5447-9s8jh" Jul 08 17:43:30.311523 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.311496 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph"] Jul 08 17:43:30.326254 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.326230 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.330799 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.330758 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:43:30.330799 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.330783 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 17:43:30.330995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.330759 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c9x66\"" Jul 08 17:43:30.330995 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.330768 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 17:43:30.335516 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.335488 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph"] Jul 08 17:43:30.336128 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.336111 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 17:43:30.336229 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.336212 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 17:43:30.411668 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.411620 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-metrics-cert\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.411668 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.411670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-manager-config\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.411912 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.411695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxrr\" (UniqueName: \"kubernetes.io/projected/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-kube-api-access-wwxrr\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.411912 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.411752 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-cert\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.512672 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.512636 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-metrics-cert\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.512814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.512683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-manager-config\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.512814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.512715 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxrr\" (UniqueName: \"kubernetes.io/projected/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-kube-api-access-wwxrr\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.512814 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.512748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-cert\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.513456 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.513432 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-manager-config\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.515284 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.515264 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-cert\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.515381 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.515313 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-metrics-cert\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.526738 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.526714 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxrr\" (UniqueName: \"kubernetes.io/projected/f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc-kube-api-access-wwxrr\") pod \"lws-controller-manager-8d6bb58cc-t9lph\" (UID: \"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc\") " pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.636869 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.636817 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:30.768850 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:30.768791 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph"] Jul 08 17:43:30.772585 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:43:30.772549 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13f4971_4cd8_45d8_a7cf_9d56f8d3a1bc.slice/crio-06568e2e3358193b2e7189cfe252679802f17e8334ccbb3cc015d231a517d12a WatchSource:0}: Error finding container 06568e2e3358193b2e7189cfe252679802f17e8334ccbb3cc015d231a517d12a: Status 404 returned error can't find the container with id 06568e2e3358193b2e7189cfe252679802f17e8334ccbb3cc015d231a517d12a Jul 08 17:43:31.328031 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:31.327987 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" event={"ID":"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc","Type":"ContainerStarted","Data":"06568e2e3358193b2e7189cfe252679802f17e8334ccbb3cc015d231a517d12a"} Jul 08 17:43:33.339894 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:33.339850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" event={"ID":"f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc","Type":"ContainerStarted","Data":"e1e87e68f7e13e175357536fde10b37f9a9b6c5fee2051290af8b9c5e3312a09"} Jul 08 17:43:33.340340 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:33.340020 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:33.395928 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:33.395876 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" podStartSLOduration=1.826540284 podStartE2EDuration="3.395860003s" totalImagesPullingTime="1.569319719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:43:33.393536033 +0000 UTC m=+585.893315772" watchObservedRunningTime="2026-07-08 17:43:33.395860003 +0000 UTC m=+585.895639737" Jul 08 17:43:40.494927 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.494894 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2"] Jul 08 17:43:40.498002 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.497982 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.500421 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.500399 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 17:43:40.500620 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.500441 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 17:43:40.500725 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.500675 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 17:43:40.500853 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.500780 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-59bzv\"" Jul 08 17:43:40.500987 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.500968 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 17:43:40.501293 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.501278 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 17:43:40.501485 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.501469 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 17:43:40.516642 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.516618 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2"] Jul 08 17:43:40.593590 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593559 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.593782 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593606 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.593782 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.593782 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.593782 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593738 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a66b758-ef3f-4e43-81d2-0d4807741157-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.593782 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593756 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.593782 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.593780 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54cs8\" (UniqueName: \"kubernetes.io/projected/2a66b758-ef3f-4e43-81d2-0d4807741157-kube-api-access-54cs8\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694569 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694531 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694574 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a66b758-ef3f-4e43-81d2-0d4807741157-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.694766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.694746 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54cs8\" (UniqueName: \"kubernetes.io/projected/2a66b758-ef3f-4e43-81d2-0d4807741157-kube-api-access-54cs8\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.695575 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.695545 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.697064 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.697035 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.697221 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.697094 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a66b758-ef3f-4e43-81d2-0d4807741157-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.697221 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.697171 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.697309 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.697269 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.704427 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.704404 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a66b758-ef3f-4e43-81d2-0d4807741157-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.705053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.705027 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54cs8\" (UniqueName: \"kubernetes.io/projected/2a66b758-ef3f-4e43-81d2-0d4807741157-kube-api-access-54cs8\") pod \"istiod-openshift-gateway-94bb8fbfd-7brb2\" (UID: \"2a66b758-ef3f-4e43-81d2-0d4807741157\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.807058 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.806969 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:40.949257 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:40.949226 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2"] Jul 08 17:43:40.952223 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:43:40.952195 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a66b758_ef3f_4e43_81d2_0d4807741157.slice/crio-8dd4d0a2b49bbb97318a2c983352e3a128b20b1e5e1edb5c3501634503d26aff WatchSource:0}: Error finding container 8dd4d0a2b49bbb97318a2c983352e3a128b20b1e5e1edb5c3501634503d26aff: Status 404 returned error can't find the container with id 8dd4d0a2b49bbb97318a2c983352e3a128b20b1e5e1edb5c3501634503d26aff Jul 08 17:43:41.376151 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:41.376109 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" event={"ID":"2a66b758-ef3f-4e43-81d2-0d4807741157","Type":"ContainerStarted","Data":"8dd4d0a2b49bbb97318a2c983352e3a128b20b1e5e1edb5c3501634503d26aff"} Jul 08 17:43:43.527935 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:43.527895 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 17:43:43.528294 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:43.527996 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 17:43:44.346795 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:44.346765 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8d6bb58cc-t9lph" Jul 08 17:43:44.393102 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:44.393057 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" event={"ID":"2a66b758-ef3f-4e43-81d2-0d4807741157","Type":"ContainerStarted","Data":"80efdaf6284ec4b51684e14008bdb04111b7dd4f8cf4691d24cb2905b38fcd4d"} Jul 08 17:43:44.393292 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:44.393221 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:44.395053 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:44.395022 2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-7brb2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 17:43:44.395168 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:44.395088 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" podUID="2a66b758-ef3f-4e43-81d2-0d4807741157" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 17:43:44.484078 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:44.484007 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" podStartSLOduration=1.910469747 podStartE2EDuration="4.483983072s" totalImagesPullingTime="2.573513325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:43:44.482689361 +0000 UTC m=+596.982469165" watchObservedRunningTime="2026-07-08 17:43:44.483983072 +0000 UTC m=+596.983762797" Jul 08 17:43:45.398066 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:45.398039 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7brb2" Jul 08 17:43:47.932571 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:47.932543 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:43:47.933035 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:43:47.932594 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:44:48.725295 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.725258 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6"] Jul 08 17:44:48.728647 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.728631 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.730955 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.730931 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-slgqj\"" Jul 08 17:44:48.731059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.730933 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:44:48.731111 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.731060 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:44:48.738136 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.738109 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6"] Jul 08 17:44:48.878985 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.878950 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.879168 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.878990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.879168 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.879018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8qx6\" (UniqueName: \"kubernetes.io/projected/212d8976-41d7-43e0-89d8-cf110169e2db-kube-api-access-d8qx6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.980476 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.980383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.980476 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.980425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.980476 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.980453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8qx6\" (UniqueName: \"kubernetes.io/projected/212d8976-41d7-43e0-89d8-cf110169e2db-kube-api-access-d8qx6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.980932 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.980911 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.981012 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.980934 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:48.988623 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:48.988598 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8qx6\" (UniqueName: \"kubernetes.io/projected/212d8976-41d7-43e0-89d8-cf110169e2db-kube-api-access-d8qx6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:49.039617 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.039570 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:49.173962 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.173926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6"] Jul 08 17:44:49.176740 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:44:49.176712 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212d8976_41d7_43e0_89d8_cf110169e2db.slice/crio-3369da8eda8b28d0b01b98635ac42583799d681eea822a4fa16ad389774f9c0a WatchSource:0}: Error finding container 3369da8eda8b28d0b01b98635ac42583799d681eea822a4fa16ad389774f9c0a: Status 404 returned error can't find the container with id 3369da8eda8b28d0b01b98635ac42583799d681eea822a4fa16ad389774f9c0a Jul 08 17:44:49.326516 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.326479 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4"] Jul 08 17:44:49.329968 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.329945 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.339613 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.339578 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4"] Jul 08 17:44:49.484857 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.484792 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.485059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.484868 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.485059 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.484985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrvlz\" (UniqueName: \"kubernetes.io/projected/073f055d-0cef-4283-ab91-ae76b15e1757-kube-api-access-xrvlz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.586128 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.586093 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrvlz\" (UniqueName: \"kubernetes.io/projected/073f055d-0cef-4283-ab91-ae76b15e1757-kube-api-access-xrvlz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.586305 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.586155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.586305 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.586174 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.586523 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.586503 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.586581 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.586566 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.595629 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.595608 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrvlz\" (UniqueName: \"kubernetes.io/projected/073f055d-0cef-4283-ab91-ae76b15e1757-kube-api-access-xrvlz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.640317 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.640290 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:49.703120 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.703087 2574 generic.go:356] "Generic (PLEG): container finished" podID="212d8976-41d7-43e0-89d8-cf110169e2db" containerID="8da504ce10f525666172cf1d5ef8e3e8d2cf1ef3761d916b3ab02b0b2727e13b" exitCode=0 Jul 08 17:44:49.703269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.703169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" event={"ID":"212d8976-41d7-43e0-89d8-cf110169e2db","Type":"ContainerDied","Data":"8da504ce10f525666172cf1d5ef8e3e8d2cf1ef3761d916b3ab02b0b2727e13b"} Jul 08 17:44:49.703269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.703215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" event={"ID":"212d8976-41d7-43e0-89d8-cf110169e2db","Type":"ContainerStarted","Data":"3369da8eda8b28d0b01b98635ac42583799d681eea822a4fa16ad389774f9c0a"} Jul 08 17:44:49.772155 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.772128 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4"] Jul 08 17:44:49.774112 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:44:49.774086 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073f055d_0cef_4283_ab91_ae76b15e1757.slice/crio-4ed7903b79019bbb975a27e96156e411e632b38c23463f1872ff507ceadf91ed WatchSource:0}: Error finding container 4ed7903b79019bbb975a27e96156e411e632b38c23463f1872ff507ceadf91ed: Status 404 returned error can't find the container with id 4ed7903b79019bbb975a27e96156e411e632b38c23463f1872ff507ceadf91ed Jul 08 17:44:49.936580 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.936549 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn"] Jul 08 17:44:49.939815 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.939797 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:49.949483 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:49.949457 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn"] Jul 08 17:44:50.089899 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.089867 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klsdg\" (UniqueName: \"kubernetes.io/projected/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-kube-api-access-klsdg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.089899 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.089903 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.090118 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.089931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.191333 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.191245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klsdg\" (UniqueName: \"kubernetes.io/projected/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-kube-api-access-klsdg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.191333 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.191297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.191553 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.191342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.191652 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.191632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.191749 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.191730 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.199760 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.199736 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klsdg\" (UniqueName: \"kubernetes.io/projected/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-kube-api-access-klsdg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.249573 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.249543 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:50.367802 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.367771 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk"] Jul 08 17:44:50.372410 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.372392 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.388013 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.387304 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk"] Jul 08 17:44:50.389646 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.389564 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn"] Jul 08 17:44:50.391985 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:44:50.391962 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a7f1fa_cbbc_4fcd_8e64_e4f8e91b77a9.slice/crio-ddfe9cafcffd0bb36af34313f952d10309535225815a9532055a8353c722433b WatchSource:0}: Error finding container ddfe9cafcffd0bb36af34313f952d10309535225815a9532055a8353c722433b: Status 404 returned error can't find the container with id ddfe9cafcffd0bb36af34313f952d10309535225815a9532055a8353c722433b Jul 08 17:44:50.494577 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.494550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gz4p\" (UniqueName: \"kubernetes.io/projected/248e88ad-4ac3-472a-8a0b-1e152e9653e5-kube-api-access-8gz4p\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.494675 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.494607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.494715 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.494680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.595939 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.595910 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.596108 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.595954 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gz4p\" (UniqueName: \"kubernetes.io/projected/248e88ad-4ac3-472a-8a0b-1e152e9653e5-kube-api-access-8gz4p\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.596108 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.595997 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.596301 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.596280 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.596341 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.596299 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.605718 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.605697 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gz4p\" (UniqueName: \"kubernetes.io/projected/248e88ad-4ac3-472a-8a0b-1e152e9653e5-kube-api-access-8gz4p\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.681808 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.681776 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:50.708470 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.708406 2574 generic.go:356] "Generic (PLEG): container finished" podID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerID="84a870b59950adc66dfdb044bca788a1b1c82170fb85878156daecef157c66a4" exitCode=0 Jul 08 17:44:50.708599 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.708491 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" event={"ID":"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9","Type":"ContainerDied","Data":"84a870b59950adc66dfdb044bca788a1b1c82170fb85878156daecef157c66a4"} Jul 08 17:44:50.708599 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.708560 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" event={"ID":"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9","Type":"ContainerStarted","Data":"ddfe9cafcffd0bb36af34313f952d10309535225815a9532055a8353c722433b"} Jul 08 17:44:50.710222 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.710191 2574 generic.go:356] "Generic (PLEG): container finished" podID="073f055d-0cef-4283-ab91-ae76b15e1757" containerID="0512486e94c18237a94387fa9a54cb02549e5bc1bd13afab26e19cb52795ca3f" exitCode=0 Jul 08 17:44:50.710350 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.710306 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" event={"ID":"073f055d-0cef-4283-ab91-ae76b15e1757","Type":"ContainerDied","Data":"0512486e94c18237a94387fa9a54cb02549e5bc1bd13afab26e19cb52795ca3f"} Jul 08 17:44:50.710393 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.710354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" event={"ID":"073f055d-0cef-4283-ab91-ae76b15e1757","Type":"ContainerStarted","Data":"4ed7903b79019bbb975a27e96156e411e632b38c23463f1872ff507ceadf91ed"} Jul 08 17:44:50.854982 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:50.854956 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk"] Jul 08 17:44:50.892631 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:44:50.892599 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248e88ad_4ac3_472a_8a0b_1e152e9653e5.slice/crio-feb0bd44d2e768519eeb300412a9dac203308183bd79c1a7a39a71b9537e39ca WatchSource:0}: Error finding container feb0bd44d2e768519eeb300412a9dac203308183bd79c1a7a39a71b9537e39ca: Status 404 returned error can't find the container with id feb0bd44d2e768519eeb300412a9dac203308183bd79c1a7a39a71b9537e39ca Jul 08 17:44:51.716490 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:51.716458 2574 generic.go:356] "Generic (PLEG): container finished" podID="212d8976-41d7-43e0-89d8-cf110169e2db" containerID="ef3d75a0146452a6f641ed60e34ced97b3c97989ea3b0c9c9561e2b526673899" exitCode=0 Jul 08 17:44:51.716670 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:51.716549 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" event={"ID":"212d8976-41d7-43e0-89d8-cf110169e2db","Type":"ContainerDied","Data":"ef3d75a0146452a6f641ed60e34ced97b3c97989ea3b0c9c9561e2b526673899"} Jul 08 17:44:51.718073 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:51.718048 2574 generic.go:356] "Generic (PLEG): container finished" podID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerID="80f200f4e8c4efbfcff4155a595a75bc4d8a9d676fa1796006a56f5eba55c930" exitCode=0 Jul 08 17:44:51.718181 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:51.718120 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" event={"ID":"248e88ad-4ac3-472a-8a0b-1e152e9653e5","Type":"ContainerDied","Data":"80f200f4e8c4efbfcff4155a595a75bc4d8a9d676fa1796006a56f5eba55c930"} Jul 08 17:44:51.718181 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:51.718157 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" event={"ID":"248e88ad-4ac3-472a-8a0b-1e152e9653e5","Type":"ContainerStarted","Data":"feb0bd44d2e768519eeb300412a9dac203308183bd79c1a7a39a71b9537e39ca"} Jul 08 17:44:52.724562 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.724535 2574 generic.go:356] "Generic (PLEG): container finished" podID="212d8976-41d7-43e0-89d8-cf110169e2db" containerID="1e12ba70b8cfa092b3ec224943154ce074f21739149fd69ff309e250e748bb07" exitCode=0 Jul 08 17:44:52.724929 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.724618 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" event={"ID":"212d8976-41d7-43e0-89d8-cf110169e2db","Type":"ContainerDied","Data":"1e12ba70b8cfa092b3ec224943154ce074f21739149fd69ff309e250e748bb07"} Jul 08 17:44:52.726236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.726215 2574 generic.go:356] "Generic (PLEG): container finished" podID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerID="d0cf90cd69d8fbc201770350c118ec8319095332a6b9b6b1a5a93da113a267b9" exitCode=0 Jul 08 17:44:52.726334 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.726290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" event={"ID":"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9","Type":"ContainerDied","Data":"d0cf90cd69d8fbc201770350c118ec8319095332a6b9b6b1a5a93da113a267b9"} Jul 08 17:44:52.727974 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.727954 2574 generic.go:356] "Generic (PLEG): container finished" podID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerID="e2cf731ac8dee9e2fc8ee2ebe1860da6f5b38a75ed838bf59b60fb0cd7ba8a5b" exitCode=0 Jul 08 17:44:52.728071 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.728028 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" event={"ID":"248e88ad-4ac3-472a-8a0b-1e152e9653e5","Type":"ContainerDied","Data":"e2cf731ac8dee9e2fc8ee2ebe1860da6f5b38a75ed838bf59b60fb0cd7ba8a5b"} Jul 08 17:44:52.729865 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.729846 2574 generic.go:356] "Generic (PLEG): container finished" podID="073f055d-0cef-4283-ab91-ae76b15e1757" containerID="3e38982865b3b0c533661657dd47233b044061ff59ba3f6698ea0e6f4547985d" exitCode=0 Jul 08 17:44:52.729925 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:52.729885 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" event={"ID":"073f055d-0cef-4283-ab91-ae76b15e1757","Type":"ContainerDied","Data":"3e38982865b3b0c533661657dd47233b044061ff59ba3f6698ea0e6f4547985d"} Jul 08 17:44:53.737017 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.736984 2574 generic.go:356] "Generic (PLEG): container finished" podID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerID="bc6295179777e7a7fb24527c53070b3a1e3510a6c62b41ec7c3ffb7c2df54722" exitCode=0 Jul 08 17:44:53.737428 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.737069 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" event={"ID":"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9","Type":"ContainerDied","Data":"bc6295179777e7a7fb24527c53070b3a1e3510a6c62b41ec7c3ffb7c2df54722"} Jul 08 17:44:53.739027 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.739002 2574 generic.go:356] "Generic (PLEG): container finished" podID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerID="94d2eda702eaecd8a160bcc27179e75b39835655ea81aa05543a3979debbc69c" exitCode=0 Jul 08 17:44:53.739149 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.739078 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" event={"ID":"248e88ad-4ac3-472a-8a0b-1e152e9653e5","Type":"ContainerDied","Data":"94d2eda702eaecd8a160bcc27179e75b39835655ea81aa05543a3979debbc69c"} Jul 08 17:44:53.741015 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.740993 2574 generic.go:356] "Generic (PLEG): container finished" podID="073f055d-0cef-4283-ab91-ae76b15e1757" containerID="cd5bf82f403df5593c2ba3ef45f7cb78d2f3c5c17b408aeea09bd815af22d498" exitCode=0 Jul 08 17:44:53.741101 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.741083 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" event={"ID":"073f055d-0cef-4283-ab91-ae76b15e1757","Type":"ContainerDied","Data":"cd5bf82f403df5593c2ba3ef45f7cb78d2f3c5c17b408aeea09bd815af22d498"} Jul 08 17:44:53.862372 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.862351 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:53.922007 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.921962 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8qx6\" (UniqueName: \"kubernetes.io/projected/212d8976-41d7-43e0-89d8-cf110169e2db-kube-api-access-d8qx6\") pod \"212d8976-41d7-43e0-89d8-cf110169e2db\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " Jul 08 17:44:53.922184 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.922029 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-bundle\") pod \"212d8976-41d7-43e0-89d8-cf110169e2db\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " Jul 08 17:44:53.922184 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.922075 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-util\") pod \"212d8976-41d7-43e0-89d8-cf110169e2db\" (UID: \"212d8976-41d7-43e0-89d8-cf110169e2db\") " Jul 08 17:44:53.922537 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.922512 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-bundle" (OuterVolumeSpecName: "bundle") pod "212d8976-41d7-43e0-89d8-cf110169e2db" (UID: "212d8976-41d7-43e0-89d8-cf110169e2db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:53.924312 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.924285 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/212d8976-41d7-43e0-89d8-cf110169e2db-kube-api-access-d8qx6" (OuterVolumeSpecName: "kube-api-access-d8qx6") pod "212d8976-41d7-43e0-89d8-cf110169e2db" (UID: "212d8976-41d7-43e0-89d8-cf110169e2db"). InnerVolumeSpecName "kube-api-access-d8qx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:44:53.927763 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:53.927738 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-util" (OuterVolumeSpecName: "util") pod "212d8976-41d7-43e0-89d8-cf110169e2db" (UID: "212d8976-41d7-43e0-89d8-cf110169e2db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:54.022937 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.022855 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:54.022937 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.022885 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/212d8976-41d7-43e0-89d8-cf110169e2db-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:54.022937 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.022916 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8qx6\" (UniqueName: \"kubernetes.io/projected/212d8976-41d7-43e0-89d8-cf110169e2db-kube-api-access-d8qx6\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:54.748115 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.748085 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" Jul 08 17:44:54.748504 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.748123 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6" event={"ID":"212d8976-41d7-43e0-89d8-cf110169e2db","Type":"ContainerDied","Data":"3369da8eda8b28d0b01b98635ac42583799d681eea822a4fa16ad389774f9c0a"} Jul 08 17:44:54.748504 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.748178 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3369da8eda8b28d0b01b98635ac42583799d681eea822a4fa16ad389774f9c0a" Jul 08 17:44:54.896349 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.896324 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:54.931221 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.931199 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klsdg\" (UniqueName: \"kubernetes.io/projected/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-kube-api-access-klsdg\") pod \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " Jul 08 17:44:54.931331 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.931267 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-util\") pod \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " Jul 08 17:44:54.931331 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.931324 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-bundle\") pod \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\" (UID: \"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9\") " Jul 08 17:44:54.931910 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.931873 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-bundle" (OuterVolumeSpecName: "bundle") pod "c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" (UID: "c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:54.932137 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.932112 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:54.933767 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.933741 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-kube-api-access-klsdg" (OuterVolumeSpecName: "kube-api-access-klsdg") pod "c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" (UID: "c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9"). InnerVolumeSpecName "kube-api-access-klsdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:44:54.936008 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.935990 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:54.936982 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:54.936956 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-util" (OuterVolumeSpecName: "util") pod "c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" (UID: "c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:55.032491 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032413 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-util\") pod \"073f055d-0cef-4283-ab91-ae76b15e1757\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " Jul 08 17:44:55.032491 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032472 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-util\") pod \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " Jul 08 17:44:55.032697 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032501 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gz4p\" (UniqueName: \"kubernetes.io/projected/248e88ad-4ac3-472a-8a0b-1e152e9653e5-kube-api-access-8gz4p\") pod \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " Jul 08 17:44:55.032697 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032541 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-bundle\") pod \"073f055d-0cef-4283-ab91-ae76b15e1757\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " Jul 08 17:44:55.032697 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032567 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrvlz\" (UniqueName: \"kubernetes.io/projected/073f055d-0cef-4283-ab91-ae76b15e1757-kube-api-access-xrvlz\") pod \"073f055d-0cef-4283-ab91-ae76b15e1757\" (UID: \"073f055d-0cef-4283-ab91-ae76b15e1757\") " Jul 08 17:44:55.032697 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032592 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-bundle\") pod \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\" (UID: \"248e88ad-4ac3-472a-8a0b-1e152e9653e5\") " Jul 08 17:44:55.032922 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032865 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klsdg\" (UniqueName: \"kubernetes.io/projected/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-kube-api-access-klsdg\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.032922 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032885 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.032922 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.032899 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.033084 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.033055 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-bundle" (OuterVolumeSpecName: "bundle") pod "073f055d-0cef-4283-ab91-ae76b15e1757" (UID: "073f055d-0cef-4283-ab91-ae76b15e1757"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:55.033409 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.033330 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-bundle" (OuterVolumeSpecName: "bundle") pod "248e88ad-4ac3-472a-8a0b-1e152e9653e5" (UID: "248e88ad-4ac3-472a-8a0b-1e152e9653e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:55.034913 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.034886 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073f055d-0cef-4283-ab91-ae76b15e1757-kube-api-access-xrvlz" (OuterVolumeSpecName: "kube-api-access-xrvlz") pod "073f055d-0cef-4283-ab91-ae76b15e1757" (UID: "073f055d-0cef-4283-ab91-ae76b15e1757"). InnerVolumeSpecName "kube-api-access-xrvlz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:44:55.035001 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.034940 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/248e88ad-4ac3-472a-8a0b-1e152e9653e5-kube-api-access-8gz4p" (OuterVolumeSpecName: "kube-api-access-8gz4p") pod "248e88ad-4ac3-472a-8a0b-1e152e9653e5" (UID: "248e88ad-4ac3-472a-8a0b-1e152e9653e5"). InnerVolumeSpecName "kube-api-access-8gz4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:44:55.038028 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.037996 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-util" (OuterVolumeSpecName: "util") pod "248e88ad-4ac3-472a-8a0b-1e152e9653e5" (UID: "248e88ad-4ac3-472a-8a0b-1e152e9653e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:55.078673 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.078629 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-util" (OuterVolumeSpecName: "util") pod "073f055d-0cef-4283-ab91-ae76b15e1757" (UID: "073f055d-0cef-4283-ab91-ae76b15e1757"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:44:55.133573 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.133548 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.133573 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.133570 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gz4p\" (UniqueName: \"kubernetes.io/projected/248e88ad-4ac3-472a-8a0b-1e152e9653e5-kube-api-access-8gz4p\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.133694 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.133580 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.133694 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.133589 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrvlz\" (UniqueName: \"kubernetes.io/projected/073f055d-0cef-4283-ab91-ae76b15e1757-kube-api-access-xrvlz\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.133694 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.133599 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/248e88ad-4ac3-472a-8a0b-1e152e9653e5-bundle\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.133694 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.133607 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/073f055d-0cef-4283-ab91-ae76b15e1757-util\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 17:44:55.754802 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.754761 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" event={"ID":"c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9","Type":"ContainerDied","Data":"ddfe9cafcffd0bb36af34313f952d10309535225815a9532055a8353c722433b"} Jul 08 17:44:55.755311 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.754820 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddfe9cafcffd0bb36af34313f952d10309535225815a9532055a8353c722433b" Jul 08 17:44:55.755311 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.754776 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn" Jul 08 17:44:55.756601 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.756571 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" Jul 08 17:44:55.756601 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.756593 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk" event={"ID":"248e88ad-4ac3-472a-8a0b-1e152e9653e5","Type":"ContainerDied","Data":"feb0bd44d2e768519eeb300412a9dac203308183bd79c1a7a39a71b9537e39ca"} Jul 08 17:44:55.756784 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.756632 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="feb0bd44d2e768519eeb300412a9dac203308183bd79c1a7a39a71b9537e39ca" Jul 08 17:44:55.758712 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.758689 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" event={"ID":"073f055d-0cef-4283-ab91-ae76b15e1757","Type":"ContainerDied","Data":"4ed7903b79019bbb975a27e96156e411e632b38c23463f1872ff507ceadf91ed"} Jul 08 17:44:55.758712 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.758706 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4" Jul 08 17:44:55.758861 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:44:55.758726 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ed7903b79019bbb975a27e96156e411e632b38c23463f1872ff507ceadf91ed" Jul 08 17:45:33.905189 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905152 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj"] Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905661 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="util" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905682 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="util" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905696 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="extract" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905706 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="extract" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905717 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="util" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905725 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="util" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905736 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="pull" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905745 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="pull" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905755 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="pull" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905763 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="pull" Jul 08 17:45:33.905769 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905776 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="util" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905785 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="util" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905792 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905798 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905808 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="pull" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905813 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="pull" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905819 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="util" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905845 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="util" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905860 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905869 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905878 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905885 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905896 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="pull" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905904 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="pull" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905976 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="248e88ad-4ac3-472a-8a0b-1e152e9653e5" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905989 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="073f055d-0cef-4283-ab91-ae76b15e1757" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.905998 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9" containerName="extract" Jul 08 17:45:33.906236 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.906008 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="212d8976-41d7-43e0-89d8-cf110169e2db" containerName="extract" Jul 08 17:45:33.909209 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.909189 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:45:33.911869 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.911847 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:45:33.912142 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.912119 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:45:33.912699 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.912679 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mt4hl\"" Jul 08 17:45:33.931737 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:33.931704 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj"] Jul 08 17:45:34.083458 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.083420 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6z2\" (UniqueName: \"kubernetes.io/projected/aed853be-8853-4140-ae7f-e947e45c4fff-kube-api-access-hm6z2\") pod \"limitador-operator-controller-manager-859d7b78bd-krckj\" (UID: \"aed853be-8853-4140-ae7f-e947e45c4fff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:45:34.184583 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.184503 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6z2\" (UniqueName: \"kubernetes.io/projected/aed853be-8853-4140-ae7f-e947e45c4fff-kube-api-access-hm6z2\") pod \"limitador-operator-controller-manager-859d7b78bd-krckj\" (UID: \"aed853be-8853-4140-ae7f-e947e45c4fff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:45:34.193361 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.193337 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6z2\" (UniqueName: \"kubernetes.io/projected/aed853be-8853-4140-ae7f-e947e45c4fff-kube-api-access-hm6z2\") pod \"limitador-operator-controller-manager-859d7b78bd-krckj\" (UID: \"aed853be-8853-4140-ae7f-e947e45c4fff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:45:34.221204 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.221167 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:45:34.376458 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.376423 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj"] Jul 08 17:45:34.380199 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:45:34.380172 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed853be_8853_4140_ae7f_e947e45c4fff.slice/crio-35579886eb9ee362411296dae1e029e0fc45231abbd1d0c367cce62c4fa6f3e1 WatchSource:0}: Error finding container 35579886eb9ee362411296dae1e029e0fc45231abbd1d0c367cce62c4fa6f3e1: Status 404 returned error can't find the container with id 35579886eb9ee362411296dae1e029e0fc45231abbd1d0c367cce62c4fa6f3e1 Jul 08 17:45:34.382453 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.382434 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:45:34.949184 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:34.949140 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" event={"ID":"aed853be-8853-4140-ae7f-e947e45c4fff","Type":"ContainerStarted","Data":"35579886eb9ee362411296dae1e029e0fc45231abbd1d0c367cce62c4fa6f3e1"} Jul 08 17:45:37.967544 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:37.967462 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" event={"ID":"aed853be-8853-4140-ae7f-e947e45c4fff","Type":"ContainerStarted","Data":"99540e384c8b8e01d0e3c6e6a3936ca4d808ace0f55660d3d7d744de40b06fad"} Jul 08 17:45:37.967945 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:37.967603 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:45:37.986504 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:37.986454 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" podStartSLOduration=2.080051399 podStartE2EDuration="4.986438539s" totalImagesPullingTime="2.90638714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:45:37.985974414 +0000 UTC m=+710.485754164" watchObservedRunningTime="2026-07-08 17:45:37.986438539 +0000 UTC m=+710.486218262" Jul 08 17:45:48.975477 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:45:48.975441 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-krckj" Jul 08 17:47:25.668328 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.668288 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gfqvm"] Jul 08 17:47:25.671438 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.671423 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.673686 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.673661 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 17:47:25.673798 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.673733 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-76h44\"" Jul 08 17:47:25.681196 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.681172 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gfqvm"] Jul 08 17:47:25.701467 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.701437 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dpdct"] Jul 08 17:47:25.704507 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.704491 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.708442 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.708419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0abacd3-e2e7-4f66-a201-3bcf597e469b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gfqvm\" (UID: \"f0abacd3-e2e7-4f66-a201-3bcf597e469b\") " pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.708559 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.708460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgppv\" (UniqueName: \"kubernetes.io/projected/f0abacd3-e2e7-4f66-a201-3bcf597e469b-kube-api-access-xgppv\") pod \"payload-pre-processing-7964457696-gfqvm\" (UID: \"f0abacd3-e2e7-4f66-a201-3bcf597e469b\") " pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.713778 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.713755 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dpdct"] Jul 08 17:47:25.809104 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.809072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0abacd3-e2e7-4f66-a201-3bcf597e469b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gfqvm\" (UID: \"f0abacd3-e2e7-4f66-a201-3bcf597e469b\") " pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.809256 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.809114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d6102f-58cd-4cc4-befc-a8ddeb457b5f-plugins-config-volume\") pod \"payload-processing-58f86c7798-dpdct\" (UID: \"49d6102f-58cd-4cc4-befc-a8ddeb457b5f\") " pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.809256 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.809138 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgppv\" (UniqueName: \"kubernetes.io/projected/f0abacd3-e2e7-4f66-a201-3bcf597e469b-kube-api-access-xgppv\") pod \"payload-pre-processing-7964457696-gfqvm\" (UID: \"f0abacd3-e2e7-4f66-a201-3bcf597e469b\") " pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.809256 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.809179 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpsj9\" (UniqueName: \"kubernetes.io/projected/49d6102f-58cd-4cc4-befc-a8ddeb457b5f-kube-api-access-gpsj9\") pod \"payload-processing-58f86c7798-dpdct\" (UID: \"49d6102f-58cd-4cc4-befc-a8ddeb457b5f\") " pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.809892 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.809872 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0abacd3-e2e7-4f66-a201-3bcf597e469b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-gfqvm\" (UID: \"f0abacd3-e2e7-4f66-a201-3bcf597e469b\") " pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.818615 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.818579 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgppv\" (UniqueName: \"kubernetes.io/projected/f0abacd3-e2e7-4f66-a201-3bcf597e469b-kube-api-access-xgppv\") pod \"payload-pre-processing-7964457696-gfqvm\" (UID: \"f0abacd3-e2e7-4f66-a201-3bcf597e469b\") " pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:25.909608 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.909576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpsj9\" (UniqueName: \"kubernetes.io/projected/49d6102f-58cd-4cc4-befc-a8ddeb457b5f-kube-api-access-gpsj9\") pod \"payload-processing-58f86c7798-dpdct\" (UID: \"49d6102f-58cd-4cc4-befc-a8ddeb457b5f\") " pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.909766 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.909681 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d6102f-58cd-4cc4-befc-a8ddeb457b5f-plugins-config-volume\") pod \"payload-processing-58f86c7798-dpdct\" (UID: \"49d6102f-58cd-4cc4-befc-a8ddeb457b5f\") " pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.910391 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.910372 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d6102f-58cd-4cc4-befc-a8ddeb457b5f-plugins-config-volume\") pod \"payload-processing-58f86c7798-dpdct\" (UID: \"49d6102f-58cd-4cc4-befc-a8ddeb457b5f\") " pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.917209 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.917184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpsj9\" (UniqueName: \"kubernetes.io/projected/49d6102f-58cd-4cc4-befc-a8ddeb457b5f-kube-api-access-gpsj9\") pod \"payload-processing-58f86c7798-dpdct\" (UID: \"49d6102f-58cd-4cc4-befc-a8ddeb457b5f\") " pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:25.982279 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:25.982204 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:26.014598 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:26.014560 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:26.121188 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:26.121160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-gfqvm"] Jul 08 17:47:26.122524 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:47:26.122489 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0abacd3_e2e7_4f66_a201_3bcf597e469b.slice/crio-76cb02ba6ca581f0e71ed9012ce8839aa6a13b683ace02f2b0ba0f187f513c9e WatchSource:0}: Error finding container 76cb02ba6ca581f0e71ed9012ce8839aa6a13b683ace02f2b0ba0f187f513c9e: Status 404 returned error can't find the container with id 76cb02ba6ca581f0e71ed9012ce8839aa6a13b683ace02f2b0ba0f187f513c9e Jul 08 17:47:26.156044 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:26.156019 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dpdct"] Jul 08 17:47:26.157507 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:47:26.157478 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d6102f_58cd_4cc4_befc_a8ddeb457b5f.slice/crio-297cf8722381bb09f58aeac628ebed4a6b6796c4eb413a8b16d9fbe7056dc01f WatchSource:0}: Error finding container 297cf8722381bb09f58aeac628ebed4a6b6796c4eb413a8b16d9fbe7056dc01f: Status 404 returned error can't find the container with id 297cf8722381bb09f58aeac628ebed4a6b6796c4eb413a8b16d9fbe7056dc01f Jul 08 17:47:26.477458 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:26.477419 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" event={"ID":"f0abacd3-e2e7-4f66-a201-3bcf597e469b","Type":"ContainerStarted","Data":"76cb02ba6ca581f0e71ed9012ce8839aa6a13b683ace02f2b0ba0f187f513c9e"} Jul 08 17:47:26.478573 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:26.478548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" event={"ID":"49d6102f-58cd-4cc4-befc-a8ddeb457b5f","Type":"ContainerStarted","Data":"297cf8722381bb09f58aeac628ebed4a6b6796c4eb413a8b16d9fbe7056dc01f"} Jul 08 17:47:28.491282 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:28.491243 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" event={"ID":"f0abacd3-e2e7-4f66-a201-3bcf597e469b","Type":"ContainerStarted","Data":"78665932a0d74eaecd395fcaf7efd94c44ab50435d744537e3ad90475d95e361"} Jul 08 17:47:28.491767 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:28.491426 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:28.492898 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:28.492867 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" event={"ID":"49d6102f-58cd-4cc4-befc-a8ddeb457b5f","Type":"ContainerStarted","Data":"de10bc11511f4ea9028aa07c68272807f0d6745f29a92defab5e3572129ea78d"} Jul 08 17:47:28.493034 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:28.492937 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:28.508740 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:28.508691 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" podStartSLOduration=2.046984389 podStartE2EDuration="3.508678202s" totalImagesPullingTime="1.461693813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:28.506624155 +0000 UTC m=+821.006403892" watchObservedRunningTime="2026-07-08 17:47:28.508678202 +0000 UTC m=+821.008457925" Jul 08 17:47:28.522723 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:28.522680 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" podStartSLOduration=2.095111443 podStartE2EDuration="3.522665993s" totalImagesPullingTime="1.42755455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:28.521485929 +0000 UTC m=+821.021265673" watchObservedRunningTime="2026-07-08 17:47:28.522665993 +0000 UTC m=+821.022445717" Jul 08 17:47:39.503017 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:39.502981 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-dpdct" Jul 08 17:47:39.503471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:39.503068 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-gfqvm" Jul 08 17:47:43.278884 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.278850 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z"] Jul 08 17:47:43.282195 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.282178 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.284763 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.284740 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 17:47:43.284954 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.284796 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 17:47:43.285388 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.285368 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zg5mg\"" Jul 08 17:47:43.285486 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.285375 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 17:47:43.294055 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.294034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z"] Jul 08 17:47:43.362269 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.362232 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.362471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.362300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.362471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.362369 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gcl9\" (UniqueName: \"kubernetes.io/projected/06b287b9-2baf-43a9-99fc-0d3c6575f52f-kube-api-access-6gcl9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.362471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.362429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.362471 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.362467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b287b9-2baf-43a9-99fc-0d3c6575f52f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.362673 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.362524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.463989 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.463948 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.463989 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.463986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464215 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gcl9\" (UniqueName: \"kubernetes.io/projected/06b287b9-2baf-43a9-99fc-0d3c6575f52f-kube-api-access-6gcl9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464215 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464215 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b287b9-2baf-43a9-99fc-0d3c6575f52f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464215 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464133 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464448 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464527 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464505 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.464580 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.464522 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.466584 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.466559 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b287b9-2baf-43a9-99fc-0d3c6575f52f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.466755 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.466734 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b287b9-2baf-43a9-99fc-0d3c6575f52f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.472876 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.472853 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gcl9\" (UniqueName: \"kubernetes.io/projected/06b287b9-2baf-43a9-99fc-0d3c6575f52f-kube-api-access-6gcl9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z\" (UID: \"06b287b9-2baf-43a9-99fc-0d3c6575f52f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.593704 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.593672 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:43.728908 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:43.728879 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z"] Jul 08 17:47:43.730886 ip-10-0-128-115 kubenswrapper[2574]: W0708 17:47:43.730857 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b287b9_2baf_43a9_99fc_0d3c6575f52f.slice/crio-d3ed85d51586678cc49201db64b611b669e4b2474a3be181622fad76bd1353b7 WatchSource:0}: Error finding container d3ed85d51586678cc49201db64b611b669e4b2474a3be181622fad76bd1353b7: Status 404 returned error can't find the container with id d3ed85d51586678cc49201db64b611b669e4b2474a3be181622fad76bd1353b7 Jul 08 17:47:44.576610 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:44.576565 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" event={"ID":"06b287b9-2baf-43a9-99fc-0d3c6575f52f","Type":"ContainerStarted","Data":"d3ed85d51586678cc49201db64b611b669e4b2474a3be181622fad76bd1353b7"} Jul 08 17:47:45.583251 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:45.583218 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" event={"ID":"06b287b9-2baf-43a9-99fc-0d3c6575f52f","Type":"ContainerStarted","Data":"57d31d2c8e65e661539d7c93191f9c00755a633889a3dc7e29b9441386ce629d"} Jul 08 17:47:45.603259 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:45.603198 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" podStartSLOduration=1.4409553609999999 podStartE2EDuration="2.60318109s" totalImagesPullingTime="1.162225729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:47:45.601393753 +0000 UTC m=+838.101173481" watchObservedRunningTime="2026-07-08 17:47:45.60318109 +0000 UTC m=+838.102960813" Jul 08 17:47:53.594591 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:53.594549 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:53.595086 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:53.594671 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:53.607257 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:53.607230 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:47:53.637100 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:47:53.637072 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z" Jul 08 17:48:47.965796 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:48:47.965761 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:48:47.967019 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:48:47.967000 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:53:47.999460 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:53:47.999429 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:53:48.002091 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:53:48.002066 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:58:48.032270 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:58:48.032238 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 17:58:48.035953 ip-10-0-128-115 kubenswrapper[2574]: I0708 17:58:48.035929 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:00:00.147030 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.146993 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9"] Jul 08 18:00:00.150244 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.150227 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:00:00.152558 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.152539 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fljv6\"" Jul 08 18:00:00.152645 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.152562 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:00:00.153289 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.153271 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:00:00.171657 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.171625 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9"] Jul 08 18:00:00.260248 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.260214 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5s46\" (UniqueName: \"kubernetes.io/projected/578f1544-d378-4da7-95fc-ac8ff00af92a-kube-api-access-b5s46\") pod \"maas-api-key-cleanup-29725560-8dlt9\" (UID: \"578f1544-d378-4da7-95fc-ac8ff00af92a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:00:00.361135 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.361102 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5s46\" (UniqueName: \"kubernetes.io/projected/578f1544-d378-4da7-95fc-ac8ff00af92a-kube-api-access-b5s46\") pod \"maas-api-key-cleanup-29725560-8dlt9\" (UID: \"578f1544-d378-4da7-95fc-ac8ff00af92a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:00:00.369344 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.369318 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5s46\" (UniqueName: \"kubernetes.io/projected/578f1544-d378-4da7-95fc-ac8ff00af92a-kube-api-access-b5s46\") pod \"maas-api-key-cleanup-29725560-8dlt9\" (UID: \"578f1544-d378-4da7-95fc-ac8ff00af92a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:00:00.460575 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.460503 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:00:00.591027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.590999 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9"] Jul 08 18:00:00.592887 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:00:00.592861 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578f1544_d378_4da7_95fc_ac8ff00af92a.slice/crio-a8a923cdb4a8d053a06e7e80a6ca5370a7bf951a415a9510e8e2961b215752b2 WatchSource:0}: Error finding container a8a923cdb4a8d053a06e7e80a6ca5370a7bf951a415a9510e8e2961b215752b2: Status 404 returned error can't find the container with id a8a923cdb4a8d053a06e7e80a6ca5370a7bf951a415a9510e8e2961b215752b2 Jul 08 18:00:00.594798 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:00.594775 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:00:01.256759 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:01.256724 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" event={"ID":"578f1544-d378-4da7-95fc-ac8ff00af92a","Type":"ContainerStarted","Data":"a8a923cdb4a8d053a06e7e80a6ca5370a7bf951a415a9510e8e2961b215752b2"} Jul 08 18:00:04.275683 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:04.275582 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" event={"ID":"578f1544-d378-4da7-95fc-ac8ff00af92a","Type":"ContainerStarted","Data":"257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83"} Jul 08 18:00:04.295474 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:00:04.295423 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" podStartSLOduration=1.00713992 podStartE2EDuration="4.295406803s" totalImagesPullingTime="3.288266883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:00:04.292995177 +0000 UTC m=+1576.792774920" watchObservedRunningTime="2026-07-08 18:00:04.295406803 +0000 UTC m=+1576.795186526" Jul 08 18:01:00.011644 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:00.011566 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9"] Jul 08 18:01:00.012089 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:00.011791 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" podUID="578f1544-d378-4da7-95fc-ac8ff00af92a" containerName="cleanup" containerID="cri-o://257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83" gracePeriod=30 Jul 08 18:01:30.187175 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.187150 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:01:30.333419 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.333386 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5s46\" (UniqueName: \"kubernetes.io/projected/578f1544-d378-4da7-95fc-ac8ff00af92a-kube-api-access-b5s46\") pod \"578f1544-d378-4da7-95fc-ac8ff00af92a\" (UID: \"578f1544-d378-4da7-95fc-ac8ff00af92a\") " Jul 08 18:01:30.335654 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.335631 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/578f1544-d378-4da7-95fc-ac8ff00af92a-kube-api-access-b5s46" (OuterVolumeSpecName: "kube-api-access-b5s46") pod "578f1544-d378-4da7-95fc-ac8ff00af92a" (UID: "578f1544-d378-4da7-95fc-ac8ff00af92a"). InnerVolumeSpecName "kube-api-access-b5s46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:01:30.434039 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.434011 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5s46\" (UniqueName: \"kubernetes.io/projected/578f1544-d378-4da7-95fc-ac8ff00af92a-kube-api-access-b5s46\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:01:30.712186 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.712097 2574 generic.go:356] "Generic (PLEG): container finished" podID="578f1544-d378-4da7-95fc-ac8ff00af92a" containerID="257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83" exitCode=137 Jul 08 18:01:30.712186 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.712146 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" Jul 08 18:01:30.712354 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.712188 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" event={"ID":"578f1544-d378-4da7-95fc-ac8ff00af92a","Type":"ContainerDied","Data":"257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83"} Jul 08 18:01:30.712354 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.712243 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9" event={"ID":"578f1544-d378-4da7-95fc-ac8ff00af92a","Type":"ContainerDied","Data":"a8a923cdb4a8d053a06e7e80a6ca5370a7bf951a415a9510e8e2961b215752b2"} Jul 08 18:01:30.712354 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.712269 2574 scope.go:117] "RemoveContainer" containerID="257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83" Jul 08 18:01:30.721639 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.721623 2574 scope.go:117] "RemoveContainer" containerID="257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83" Jul 08 18:01:30.721924 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:01:30.721902 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83\": container with ID starting with 257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83 not found: ID does not exist" containerID="257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83" Jul 08 18:01:30.721971 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.721933 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83"} err="failed to get container status \"257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83\": rpc error: code = NotFound desc = could not find container \"257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83\": container with ID starting with 257e665949e7bcceaf396301819c23ca1761321d225d4c6c43f24e8efac93b83 not found: ID does not exist" Jul 08 18:01:30.735808 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.735784 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9"] Jul 08 18:01:30.748907 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:30.748882 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725560-8dlt9"] Jul 08 18:01:32.064802 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:01:32.064768 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="578f1544-d378-4da7-95fc-ac8ff00af92a" path="/var/lib/kubelet/pods/578f1544-d378-4da7-95fc-ac8ff00af92a/volumes" Jul 08 18:03:48.072299 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:03:48.072263 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:03:48.078018 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:03:48.077995 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:06:02.876967 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.876935 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5"] Jul 08 18:06:02.877408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.877256 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578f1544-d378-4da7-95fc-ac8ff00af92a" containerName="cleanup" Jul 08 18:06:02.877408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.877267 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578f1544-d378-4da7-95fc-ac8ff00af92a" containerName="cleanup" Jul 08 18:06:02.877408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.877336 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="578f1544-d378-4da7-95fc-ac8ff00af92a" containerName="cleanup" Jul 08 18:06:02.880288 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.880267 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:02.882605 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.882579 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2d05ecf5-serving-cert\"" Jul 08 18:06:02.882725 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.882666 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fljv6\"" Jul 08 18:06:02.882725 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.882685 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:06:02.883446 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.883426 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:06:02.897762 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.897740 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5"] Jul 08 18:06:02.925547 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.925520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-maas-api-tls\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:02.925645 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.925587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmcd9\" (UniqueName: \"kubernetes.io/projected/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-kube-api-access-nmcd9\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:02.925717 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:02.925695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-openshift-service-ca\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.026287 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.026256 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmcd9\" (UniqueName: \"kubernetes.io/projected/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-kube-api-access-nmcd9\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.026430 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.026321 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-openshift-service-ca\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.026430 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.026348 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-maas-api-tls\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.027109 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.027091 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-openshift-service-ca\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.028899 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.028883 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-maas-api-tls\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.035449 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.035414 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmcd9\" (UniqueName: \"kubernetes.io/projected/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-kube-api-access-nmcd9\") pod \"maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.191909 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.191794 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:03.322219 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.322189 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5"] Jul 08 18:06:03.324204 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:06:03.324176 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff95631_a586_49f0_b04c_fdb6a0f4d37f.slice/crio-8323d3cb4051880bb7467b97db0b949634b76bfc30bb216ccdba50516e16c47e WatchSource:0}: Error finding container 8323d3cb4051880bb7467b97db0b949634b76bfc30bb216ccdba50516e16c47e: Status 404 returned error can't find the container with id 8323d3cb4051880bb7467b97db0b949634b76bfc30bb216ccdba50516e16c47e Jul 08 18:06:03.325654 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:03.325626 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:06:04.077222 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:04.077179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" event={"ID":"4ff95631-a586-49f0-b04c-fdb6a0f4d37f","Type":"ContainerStarted","Data":"8323d3cb4051880bb7467b97db0b949634b76bfc30bb216ccdba50516e16c47e"} Jul 08 18:06:05.084442 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:05.084399 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" event={"ID":"4ff95631-a586-49f0-b04c-fdb6a0f4d37f","Type":"ContainerStarted","Data":"7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137"} Jul 08 18:06:05.084899 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:05.084471 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:05.104069 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:05.103937 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" podStartSLOduration=1.527595778 podStartE2EDuration="3.103916996s" totalImagesPullingTime="1.576321218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:06:05.103310598 +0000 UTC m=+1937.603090339" watchObservedRunningTime="2026-07-08 18:06:05.103916996 +0000 UTC m=+1937.603696718" Jul 08 18:06:11.094844 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:11.094799 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:19.858077 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.858035 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q"] Jul 08 18:06:19.861488 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.861465 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:19.865556 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.865534 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2d05ecf5\"/\"default-dockercfg-6m2sn\"" Jul 08 18:06:19.865664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.865568 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2d05ecf5\"/\"e2e-lifecycle-model-2d05ecf5-kserve-self-signed-certs\"" Jul 08 18:06:19.865664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.865573 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2d05ecf5\"/\"kube-root-ca.crt\"" Jul 08 18:06:19.865775 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.865534 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2d05ecf5\"/\"openshift-service-ca.crt\"" Jul 08 18:06:19.874944 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.874922 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q"] Jul 08 18:06:19.977841 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.977795 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tls-certs\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:19.978036 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.977919 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-dshm\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:19.978036 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.977947 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-model-cache\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:19.978036 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.977970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tmp-dir\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:19.978036 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.978013 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-home\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:19.978178 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:19.978070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjr7v\" (UniqueName: \"kubernetes.io/projected/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-kube-api-access-jjr7v\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079210 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tmp-dir\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079405 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079238 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-home\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079405 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjr7v\" (UniqueName: \"kubernetes.io/projected/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-kube-api-access-jjr7v\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079405 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079333 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tls-certs\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079405 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079377 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-dshm\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079405 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079399 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-model-cache\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079653 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079592 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tmp-dir\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079653 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079622 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-home\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.079794 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.079776 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-model-cache\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.081870 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.081818 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-dshm\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.082030 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.082013 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tls-certs\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.088992 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.088972 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjr7v\" (UniqueName: \"kubernetes.io/projected/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-kube-api-access-jjr7v\") pod \"e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.172027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.171944 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:20.305623 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:20.305593 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q"] Jul 08 18:06:20.307684 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:06:20.307658 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee94eebb_0069_4b8c_9e35_1b2ab3a3b1b5.slice/crio-ed50ce232af01e19103c31c1d1e6b78b80d3176ebffe6cd4b42a4bbb35513453 WatchSource:0}: Error finding container ed50ce232af01e19103c31c1d1e6b78b80d3176ebffe6cd4b42a4bbb35513453: Status 404 returned error can't find the container with id ed50ce232af01e19103c31c1d1e6b78b80d3176ebffe6cd4b42a4bbb35513453 Jul 08 18:06:21.167275 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:21.167240 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" event={"ID":"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5","Type":"ContainerStarted","Data":"a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d"} Jul 08 18:06:21.167648 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:21.167282 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" event={"ID":"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5","Type":"ContainerStarted","Data":"ed50ce232af01e19103c31c1d1e6b78b80d3176ebffe6cd4b42a4bbb35513453"} Jul 08 18:06:21.185172 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:21.185127 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" podStartSLOduration=2.1851131759999998 podStartE2EDuration="2.185113176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:06:21.183672058 +0000 UTC m=+1953.683451796" watchObservedRunningTime="2026-07-08 18:06:21.185113176 +0000 UTC m=+1953.684892898" Jul 08 18:06:30.172568 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:30.172533 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:30.174977 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:30.172587 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:30.184731 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:30.184708 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:30.225203 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:30.225171 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:06:45.251430 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.251347 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5"] Jul 08 18:06:45.252026 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.251599 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" podUID="4ff95631-a586-49f0-b04c-fdb6a0f4d37f" containerName="maas-api" containerID="cri-o://7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137" gracePeriod=30 Jul 08 18:06:45.500021 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.499997 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:45.605747 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.605722 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-openshift-service-ca\") pod \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " Jul 08 18:06:45.605940 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.605765 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmcd9\" (UniqueName: \"kubernetes.io/projected/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-kube-api-access-nmcd9\") pod \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " Jul 08 18:06:45.605940 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.605787 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-maas-api-tls\") pod \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\" (UID: \"4ff95631-a586-49f0-b04c-fdb6a0f4d37f\") " Jul 08 18:06:45.606146 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.606110 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4ff95631-a586-49f0-b04c-fdb6a0f4d37f" (UID: "4ff95631-a586-49f0-b04c-fdb6a0f4d37f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:06:45.608014 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.607989 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4ff95631-a586-49f0-b04c-fdb6a0f4d37f" (UID: "4ff95631-a586-49f0-b04c-fdb6a0f4d37f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:06:45.608115 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.608097 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-kube-api-access-nmcd9" (OuterVolumeSpecName: "kube-api-access-nmcd9") pod "4ff95631-a586-49f0-b04c-fdb6a0f4d37f" (UID: "4ff95631-a586-49f0-b04c-fdb6a0f4d37f"). InnerVolumeSpecName "kube-api-access-nmcd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:06:45.707365 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.707315 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:06:45.707365 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.707359 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmcd9\" (UniqueName: \"kubernetes.io/projected/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-kube-api-access-nmcd9\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:06:45.707365 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:45.707373 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ff95631-a586-49f0-b04c-fdb6a0f4d37f-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:06:46.294480 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.294443 2574 generic.go:356] "Generic (PLEG): container finished" podID="4ff95631-a586-49f0-b04c-fdb6a0f4d37f" containerID="7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137" exitCode=0 Jul 08 18:06:46.294929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.294503 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" Jul 08 18:06:46.294929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.294512 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" event={"ID":"4ff95631-a586-49f0-b04c-fdb6a0f4d37f","Type":"ContainerDied","Data":"7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137"} Jul 08 18:06:46.294929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.294563 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5" event={"ID":"4ff95631-a586-49f0-b04c-fdb6a0f4d37f","Type":"ContainerDied","Data":"8323d3cb4051880bb7467b97db0b949634b76bfc30bb216ccdba50516e16c47e"} Jul 08 18:06:46.294929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.294587 2574 scope.go:117] "RemoveContainer" containerID="7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137" Jul 08 18:06:46.303711 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.303692 2574 scope.go:117] "RemoveContainer" containerID="7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137" Jul 08 18:06:46.304000 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:06:46.303982 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137\": container with ID starting with 7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137 not found: ID does not exist" containerID="7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137" Jul 08 18:06:46.304059 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.304008 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137"} err="failed to get container status \"7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137\": rpc error: code = NotFound desc = could not find container \"7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137\": container with ID starting with 7a35467d5132c60f492d11c29b448dfa4fd7411856727572e1aadea108e6f137 not found: ID does not exist" Jul 08 18:06:46.311724 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.311697 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5"] Jul 08 18:06:46.315156 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:46.315132 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2d05ecf5-6cfcf4c8b4-wgnl5"] Jul 08 18:06:48.064061 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:48.064030 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ff95631-a586-49f0-b04c-fdb6a0f4d37f" path="/var/lib/kubelet/pods/4ff95631-a586-49f0-b04c-fdb6a0f4d37f/volumes" Jul 08 18:06:55.992891 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:06:55.992851 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q"] Jul 08 18:07:11.013463 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.013395 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" podUID="ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" containerName="main" containerID="cri-o://a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d" gracePeriod=45 Jul 08 18:07:11.263552 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.263504 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:07:11.337759 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.337731 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tls-certs\") pod \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " Jul 08 18:07:11.337942 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.337887 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-dshm\") pod \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " Jul 08 18:07:11.337942 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.337913 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-model-cache\") pod \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " Jul 08 18:07:11.338066 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.337946 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjr7v\" (UniqueName: \"kubernetes.io/projected/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-kube-api-access-jjr7v\") pod \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " Jul 08 18:07:11.338066 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.337983 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tmp-dir\") pod \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " Jul 08 18:07:11.338066 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.338012 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-home\") pod \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\" (UID: \"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5\") " Jul 08 18:07:11.338201 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.338141 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" (UID: "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:11.338292 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.338278 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-model-cache\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:07:11.338351 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.338284 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" (UID: "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:11.338351 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.338298 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-home" (OuterVolumeSpecName: "home") pod "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" (UID: "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:11.340082 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.340060 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-dshm" (OuterVolumeSpecName: "dshm") pod "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" (UID: "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:11.340162 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.340146 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" (UID: "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:11.340225 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.340201 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-kube-api-access-jjr7v" (OuterVolumeSpecName: "kube-api-access-jjr7v") pod "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" (UID: "ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5"). InnerVolumeSpecName "kube-api-access-jjr7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:07:11.420362 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.420329 2574 generic.go:356] "Generic (PLEG): container finished" podID="ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" containerID="a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d" exitCode=0 Jul 08 18:07:11.420522 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.420381 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" Jul 08 18:07:11.420522 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.420386 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" event={"ID":"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5","Type":"ContainerDied","Data":"a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d"} Jul 08 18:07:11.420522 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.420422 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q" event={"ID":"ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5","Type":"ContainerDied","Data":"ed50ce232af01e19103c31c1d1e6b78b80d3176ebffe6cd4b42a4bbb35513453"} Jul 08 18:07:11.420522 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.420442 2574 scope.go:117] "RemoveContainer" containerID="a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d" Jul 08 18:07:11.429848 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.429808 2574 scope.go:117] "RemoveContainer" containerID="a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d" Jul 08 18:07:11.430112 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:07:11.430093 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d\": container with ID starting with a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d not found: ID does not exist" containerID="a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d" Jul 08 18:07:11.430174 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.430125 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d"} err="failed to get container status \"a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d\": rpc error: code = NotFound desc = could not find container \"a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d\": container with ID starting with a634e39e5066d4124a55f9bce8e3d8acf87c7889ac6a4cbdb80ba0ec9002603d not found: ID does not exist" Jul 08 18:07:11.439290 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.439264 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-home\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:07:11.439457 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.439292 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tls-certs\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:07:11.439457 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.439303 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-dshm\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:07:11.439457 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.439312 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjr7v\" (UniqueName: \"kubernetes.io/projected/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-kube-api-access-jjr7v\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:07:11.439457 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.439339 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5-tmp-dir\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:07:11.443365 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.443337 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q"] Jul 08 18:07:11.445300 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:11.445279 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2d05ecf5/e2e-lifecycle-model-2d05ecf5-kserve-799cb4955-cg79q"] Jul 08 18:07:12.064010 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:07:12.063978 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" path="/var/lib/kubelet/pods/ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5/volumes" Jul 08 18:08:40.670918 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.670883 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4"] Jul 08 18:08:40.671341 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.671228 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff95631-a586-49f0-b04c-fdb6a0f4d37f" containerName="maas-api" Jul 08 18:08:40.671341 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.671240 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff95631-a586-49f0-b04c-fdb6a0f4d37f" containerName="maas-api" Jul 08 18:08:40.671341 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.671258 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" containerName="main" Jul 08 18:08:40.671341 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.671264 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" containerName="main" Jul 08 18:08:40.671341 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.671317 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff95631-a586-49f0-b04c-fdb6a0f4d37f" containerName="maas-api" Jul 08 18:08:40.671341 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.671325 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee94eebb-0069-4b8c-9e35-1b2ab3a3b1b5" containerName="main" Jul 08 18:08:40.673916 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.673901 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.676370 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.676340 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a359d5-serving-cert\"" Jul 08 18:08:40.676490 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.676372 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:08:40.676490 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.676421 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:08:40.682889 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.677267 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fljv6\"" Jul 08 18:08:40.686488 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.686461 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4"] Jul 08 18:08:40.799948 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.799915 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.800141 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.799957 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b57c094-1fab-45c5-a7a4-090648374589-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.800141 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.799994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx6tm\" (UniqueName: \"kubernetes.io/projected/6b57c094-1fab-45c5-a7a4-090648374589-kube-api-access-bx6tm\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.900701 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.900663 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.900701 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.900706 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b57c094-1fab-45c5-a7a4-090648374589-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.900965 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.900735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx6tm\" (UniqueName: \"kubernetes.io/projected/6b57c094-1fab-45c5-a7a4-090648374589-kube-api-access-bx6tm\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.900965 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:08:40.900807 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-serving-cert: secret "maas-api-e2e-inf-a-a359d5-serving-cert" not found Jul 08 18:08:40.900965 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:08:40.900898 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls podName:6b57c094-1fab-45c5-a7a4-090648374589 nodeName:}" failed. No retries permitted until 2026-07-08 18:08:41.40087975 +0000 UTC m=+2093.900659451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls") pod "maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" (UID: "6b57c094-1fab-45c5-a7a4-090648374589") : secret "maas-api-e2e-inf-a-a359d5-serving-cert" not found Jul 08 18:08:40.901346 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.901327 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b57c094-1fab-45c5-a7a4-090648374589-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:40.915178 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:40.915157 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx6tm\" (UniqueName: \"kubernetes.io/projected/6b57c094-1fab-45c5-a7a4-090648374589-kube-api-access-bx6tm\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:41.404707 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:41.404672 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:41.407181 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:41.407156 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls\") pod \"maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:41.592523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:41.592483 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:41.732554 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:41.732525 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4"] Jul 08 18:08:41.733917 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:08:41.733876 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b57c094_1fab_45c5_a7a4_090648374589.slice/crio-2f6e2ae50d7574f3ca224c14166fc9da87a81f609bf4f68ab0b711d9ac4ed182 WatchSource:0}: Error finding container 2f6e2ae50d7574f3ca224c14166fc9da87a81f609bf4f68ab0b711d9ac4ed182: Status 404 returned error can't find the container with id 2f6e2ae50d7574f3ca224c14166fc9da87a81f609bf4f68ab0b711d9ac4ed182 Jul 08 18:08:41.873029 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:41.872989 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" event={"ID":"6b57c094-1fab-45c5-a7a4-090648374589","Type":"ContainerStarted","Data":"2f6e2ae50d7574f3ca224c14166fc9da87a81f609bf4f68ab0b711d9ac4ed182"} Jul 08 18:08:42.879570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:42.879474 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" event={"ID":"6b57c094-1fab-45c5-a7a4-090648374589","Type":"ContainerStarted","Data":"598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95"} Jul 08 18:08:42.879934 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:42.879595 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:42.896856 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:42.896795 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" podStartSLOduration=2.03700695 podStartE2EDuration="2.896777963s" totalImagesPullingTime="859.771013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:08:42.895674556 +0000 UTC m=+2095.395454278" watchObservedRunningTime="2026-07-08 18:08:42.896777963 +0000 UTC m=+2095.396557687" Jul 08 18:08:48.104520 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.104488 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:08:48.111750 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.111723 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:08:48.767427 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.767395 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h"] Jul 08 18:08:48.771047 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.771026 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.773588 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.773567 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c101a9-serving-cert\"" Jul 08 18:08:48.783334 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.783308 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h"] Jul 08 18:08:48.874365 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.874328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkj7r\" (UniqueName: \"kubernetes.io/projected/f123c04d-0210-4307-b6c7-1f25de41c2a2-kube-api-access-jkj7r\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.874557 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.874374 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f123c04d-0210-4307-b6c7-1f25de41c2a2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.874557 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.874394 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.891526 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.891501 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:08:48.975009 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.974965 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkj7r\" (UniqueName: \"kubernetes.io/projected/f123c04d-0210-4307-b6c7-1f25de41c2a2-kube-api-access-jkj7r\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.975209 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.975060 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f123c04d-0210-4307-b6c7-1f25de41c2a2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.975209 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.975090 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.975330 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:08:48.975264 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-serving-cert: secret "maas-api-e2e-inf-b-c101a9-serving-cert" not found Jul 08 18:08:48.975392 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:08:48.975345 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls podName:f123c04d-0210-4307-b6c7-1f25de41c2a2 nodeName:}" failed. No retries permitted until 2026-07-08 18:08:49.475320868 +0000 UTC m=+2101.975100578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls") pod "maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" (UID: "f123c04d-0210-4307-b6c7-1f25de41c2a2") : secret "maas-api-e2e-inf-b-c101a9-serving-cert" not found Jul 08 18:08:48.975953 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.975933 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f123c04d-0210-4307-b6c7-1f25de41c2a2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:48.988579 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:48.988553 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkj7r\" (UniqueName: \"kubernetes.io/projected/f123c04d-0210-4307-b6c7-1f25de41c2a2-kube-api-access-jkj7r\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:49.480946 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:49.480909 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:49.483500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:49.483470 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:49.682354 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:49.682312 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:49.819289 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:49.819216 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h"] Jul 08 18:08:49.821388 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:08:49.821356 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf123c04d_0210_4307_b6c7_1f25de41c2a2.slice/crio-21e8c5b86751f96601b1ff8806fc3192c273930cfdf89c7643f93170b89e8662 WatchSource:0}: Error finding container 21e8c5b86751f96601b1ff8806fc3192c273930cfdf89c7643f93170b89e8662: Status 404 returned error can't find the container with id 21e8c5b86751f96601b1ff8806fc3192c273930cfdf89c7643f93170b89e8662 Jul 08 18:08:49.923761 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:49.923722 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" event={"ID":"f123c04d-0210-4307-b6c7-1f25de41c2a2","Type":"ContainerStarted","Data":"21e8c5b86751f96601b1ff8806fc3192c273930cfdf89c7643f93170b89e8662"} Jul 08 18:08:50.929968 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:50.929918 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" event={"ID":"f123c04d-0210-4307-b6c7-1f25de41c2a2","Type":"ContainerStarted","Data":"f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a"} Jul 08 18:08:50.930354 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:50.930043 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:08:50.954960 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:50.951377 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" podStartSLOduration=2.550393478 podStartE2EDuration="2.95135016s" totalImagesPullingTime="400.956682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:08:50.948358784 +0000 UTC m=+2103.448138508" watchObservedRunningTime="2026-07-08 18:08:50.95135016 +0000 UTC m=+2103.451129885" Jul 08 18:08:56.940686 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:08:56.940652 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:09:05.888365 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.888325 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5"] Jul 08 18:09:05.892023 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.892006 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:05.895216 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.895192 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a359d5\"/\"test-model-a359d5-kserve-self-signed-certs\"" Jul 08 18:09:05.895216 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.895208 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a359d5\"/\"openshift-service-ca.crt\"" Jul 08 18:09:05.895216 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.895219 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a359d5\"/\"kube-root-ca.crt\"" Jul 08 18:09:05.895463 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.895296 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a359d5\"/\"default-dockercfg-7q5md\"" Jul 08 18:09:05.917325 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917292 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5"] Jul 08 18:09:05.917594 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917569 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjl9m\" (UniqueName: \"kubernetes.io/projected/a71998aa-bdd9-4931-85b1-bcc0d4160f36-kube-api-access-vjl9m\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:05.917680 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917611 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tmp-dir\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:05.917680 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tls-certs\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:05.917792 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917683 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-dshm\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:05.917792 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-model-cache\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:05.917927 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:05.917796 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-home\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.018515 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018481 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjl9m\" (UniqueName: \"kubernetes.io/projected/a71998aa-bdd9-4931-85b1-bcc0d4160f36-kube-api-access-vjl9m\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.018695 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018522 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tmp-dir\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.018695 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tls-certs\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.018695 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018557 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-dshm\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.018695 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018582 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-model-cache\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.018695 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018620 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-home\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.019045 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.018963 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tmp-dir\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.019082 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.019055 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-home\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.019117 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.019087 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-model-cache\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.021078 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.021054 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-dshm\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.021159 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.021111 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tls-certs\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.028101 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.028078 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjl9m\" (UniqueName: \"kubernetes.io/projected/a71998aa-bdd9-4931-85b1-bcc0d4160f36-kube-api-access-vjl9m\") pod \"test-model-a359d5-kserve-54cf45c79f-xg8f5\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.202486 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.202395 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:06.336723 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:06.336700 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5"] Jul 08 18:09:06.338366 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:09:06.338339 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda71998aa_bdd9_4931_85b1_bcc0d4160f36.slice/crio-81a78093e370f32d2b5d152e5b102740185a4f129c045e899962bf457925ec33 WatchSource:0}: Error finding container 81a78093e370f32d2b5d152e5b102740185a4f129c045e899962bf457925ec33: Status 404 returned error can't find the container with id 81a78093e370f32d2b5d152e5b102740185a4f129c045e899962bf457925ec33 Jul 08 18:09:07.012134 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:07.012094 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" event={"ID":"a71998aa-bdd9-4931-85b1-bcc0d4160f36","Type":"ContainerStarted","Data":"1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c"} Jul 08 18:09:07.012516 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:07.012148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" event={"ID":"a71998aa-bdd9-4931-85b1-bcc0d4160f36","Type":"ContainerStarted","Data":"81a78093e370f32d2b5d152e5b102740185a4f129c045e899962bf457925ec33"} Jul 08 18:09:07.031697 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:07.031649 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" podStartSLOduration=2.031634638 podStartE2EDuration="2.031634638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:09:07.029211279 +0000 UTC m=+2119.528991006" watchObservedRunningTime="2026-07-08 18:09:07.031634638 +0000 UTC m=+2119.531414362" Jul 08 18:09:16.202520 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:16.202482 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:16.202520 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:16.202527 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:16.215140 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:16.215112 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:17.076085 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:17.076056 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:09:57.050240 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:57.050158 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5"] Jul 08 18:09:58.819695 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:58.819659 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4"] Jul 08 18:09:58.820144 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:58.819947 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" podUID="6b57c094-1fab-45c5-a7a4-090648374589" containerName="maas-api" containerID="cri-o://598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95" gracePeriod=30 Jul 08 18:09:58.885523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:58.885484 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" podUID="6b57c094-1fab-45c5-a7a4-090648374589" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.36:8443/health\": dial tcp 10.134.0.36:8443: connect: connection refused" Jul 08 18:09:59.062083 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.062058 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:09:59.104589 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.104563 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx6tm\" (UniqueName: \"kubernetes.io/projected/6b57c094-1fab-45c5-a7a4-090648374589-kube-api-access-bx6tm\") pod \"6b57c094-1fab-45c5-a7a4-090648374589\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " Jul 08 18:09:59.104744 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.104632 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls\") pod \"6b57c094-1fab-45c5-a7a4-090648374589\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " Jul 08 18:09:59.104744 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.104675 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b57c094-1fab-45c5-a7a4-090648374589-openshift-service-ca\") pod \"6b57c094-1fab-45c5-a7a4-090648374589\" (UID: \"6b57c094-1fab-45c5-a7a4-090648374589\") " Jul 08 18:09:59.105158 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.105123 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b57c094-1fab-45c5-a7a4-090648374589-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b57c094-1fab-45c5-a7a4-090648374589" (UID: "6b57c094-1fab-45c5-a7a4-090648374589"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:09:59.106799 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.106778 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b57c094-1fab-45c5-a7a4-090648374589-kube-api-access-bx6tm" (OuterVolumeSpecName: "kube-api-access-bx6tm") pod "6b57c094-1fab-45c5-a7a4-090648374589" (UID: "6b57c094-1fab-45c5-a7a4-090648374589"). InnerVolumeSpecName "kube-api-access-bx6tm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:09:59.106984 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.106943 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b57c094-1fab-45c5-a7a4-090648374589" (UID: "6b57c094-1fab-45c5-a7a4-090648374589"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:09:59.205420 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.205381 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx6tm\" (UniqueName: \"kubernetes.io/projected/6b57c094-1fab-45c5-a7a4-090648374589-kube-api-access-bx6tm\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:09:59.205420 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.205412 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b57c094-1fab-45c5-a7a4-090648374589-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:09:59.205420 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.205422 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b57c094-1fab-45c5-a7a4-090648374589-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:09:59.279577 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.279537 2574 generic.go:356] "Generic (PLEG): container finished" podID="6b57c094-1fab-45c5-a7a4-090648374589" containerID="598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95" exitCode=0 Jul 08 18:09:59.279577 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.279583 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" event={"ID":"6b57c094-1fab-45c5-a7a4-090648374589","Type":"ContainerDied","Data":"598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95"} Jul 08 18:09:59.279816 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.279606 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" Jul 08 18:09:59.279816 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.279648 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4" event={"ID":"6b57c094-1fab-45c5-a7a4-090648374589","Type":"ContainerDied","Data":"2f6e2ae50d7574f3ca224c14166fc9da87a81f609bf4f68ab0b711d9ac4ed182"} Jul 08 18:09:59.279816 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.279670 2574 scope.go:117] "RemoveContainer" containerID="598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95" Jul 08 18:09:59.289141 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.289123 2574 scope.go:117] "RemoveContainer" containerID="598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95" Jul 08 18:09:59.289415 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:09:59.289392 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95\": container with ID starting with 598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95 not found: ID does not exist" containerID="598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95" Jul 08 18:09:59.289462 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.289423 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95"} err="failed to get container status \"598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95\": rpc error: code = NotFound desc = could not find container \"598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95\": container with ID starting with 598f606e12242b20f8ca404d7f38eb94d9f050dbff073c0d43a1d386fc57ac95 not found: ID does not exist" Jul 08 18:09:59.300645 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.300619 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4"] Jul 08 18:09:59.304782 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:09:59.304760 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a359d5-77b9cfffff-88ws4"] Jul 08 18:10:00.064052 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:00.064018 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b57c094-1fab-45c5-a7a4-090648374589" path="/var/lib/kubelet/pods/6b57c094-1fab-45c5-a7a4-090648374589/volumes" Jul 08 18:10:12.075918 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.075878 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" podUID="a71998aa-bdd9-4931-85b1-bcc0d4160f36" containerName="main" containerID="cri-o://1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c" gracePeriod=45 Jul 08 18:10:12.322434 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.322411 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:10:12.348529 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.348459 2574 generic.go:356] "Generic (PLEG): container finished" podID="a71998aa-bdd9-4931-85b1-bcc0d4160f36" containerID="1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c" exitCode=0 Jul 08 18:10:12.348529 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.348503 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" Jul 08 18:10:12.348722 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.348548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" event={"ID":"a71998aa-bdd9-4931-85b1-bcc0d4160f36","Type":"ContainerDied","Data":"1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c"} Jul 08 18:10:12.348722 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.348594 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5" event={"ID":"a71998aa-bdd9-4931-85b1-bcc0d4160f36","Type":"ContainerDied","Data":"81a78093e370f32d2b5d152e5b102740185a4f129c045e899962bf457925ec33"} Jul 08 18:10:12.348722 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.348619 2574 scope.go:117] "RemoveContainer" containerID="1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c" Jul 08 18:10:12.357159 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.357143 2574 scope.go:117] "RemoveContainer" containerID="1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c" Jul 08 18:10:12.357431 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:10:12.357410 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c\": container with ID starting with 1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c not found: ID does not exist" containerID="1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c" Jul 08 18:10:12.357489 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.357440 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c"} err="failed to get container status \"1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c\": rpc error: code = NotFound desc = could not find container \"1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c\": container with ID starting with 1fbb08bb5320cfcfe941564dcd2b021bc13ae200898290608791a384a464408c not found: ID does not exist" Jul 08 18:10:12.420748 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.420715 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-model-cache\") pod \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " Jul 08 18:10:12.420946 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.420762 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-home\") pod \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " Jul 08 18:10:12.420946 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.420792 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjl9m\" (UniqueName: \"kubernetes.io/projected/a71998aa-bdd9-4931-85b1-bcc0d4160f36-kube-api-access-vjl9m\") pod \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " Jul 08 18:10:12.420946 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.420885 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-dshm\") pod \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " Jul 08 18:10:12.420946 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.420934 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tmp-dir\") pod \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " Jul 08 18:10:12.421153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.420983 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tls-certs\") pod \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\" (UID: \"a71998aa-bdd9-4931-85b1-bcc0d4160f36\") " Jul 08 18:10:12.421153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.421071 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-home" (OuterVolumeSpecName: "home") pod "a71998aa-bdd9-4931-85b1-bcc0d4160f36" (UID: "a71998aa-bdd9-4931-85b1-bcc0d4160f36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:10:12.421153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.421083 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-model-cache" (OuterVolumeSpecName: "model-cache") pod "a71998aa-bdd9-4931-85b1-bcc0d4160f36" (UID: "a71998aa-bdd9-4931-85b1-bcc0d4160f36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:10:12.421271 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.421229 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a71998aa-bdd9-4931-85b1-bcc0d4160f36" (UID: "a71998aa-bdd9-4931-85b1-bcc0d4160f36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:10:12.421383 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.421299 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-model-cache\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:12.421383 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.421318 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-home\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:12.423217 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.423192 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71998aa-bdd9-4931-85b1-bcc0d4160f36-kube-api-access-vjl9m" (OuterVolumeSpecName: "kube-api-access-vjl9m") pod "a71998aa-bdd9-4931-85b1-bcc0d4160f36" (UID: "a71998aa-bdd9-4931-85b1-bcc0d4160f36"). InnerVolumeSpecName "kube-api-access-vjl9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:10:12.423314 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.423249 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-dshm" (OuterVolumeSpecName: "dshm") pod "a71998aa-bdd9-4931-85b1-bcc0d4160f36" (UID: "a71998aa-bdd9-4931-85b1-bcc0d4160f36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:10:12.423314 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.423250 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a71998aa-bdd9-4931-85b1-bcc0d4160f36" (UID: "a71998aa-bdd9-4931-85b1-bcc0d4160f36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:10:12.522462 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.522433 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tls-certs\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:12.522462 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.522460 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjl9m\" (UniqueName: \"kubernetes.io/projected/a71998aa-bdd9-4931-85b1-bcc0d4160f36-kube-api-access-vjl9m\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:12.522462 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.522471 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-dshm\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:12.522687 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.522480 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a71998aa-bdd9-4931-85b1-bcc0d4160f36-tmp-dir\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:12.670267 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.670236 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5"] Jul 08 18:10:12.674061 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:12.674032 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a359d5/test-model-a359d5-kserve-54cf45c79f-xg8f5"] Jul 08 18:10:14.064228 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:14.064193 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71998aa-bdd9-4931-85b1-bcc0d4160f36" path="/var/lib/kubelet/pods/a71998aa-bdd9-4931-85b1-bcc0d4160f36/volumes" Jul 08 18:10:42.506810 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.506779 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h"] Jul 08 18:10:42.507302 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.507016 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" podUID="f123c04d-0210-4307-b6c7-1f25de41c2a2" containerName="maas-api" containerID="cri-o://f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a" gracePeriod=30 Jul 08 18:10:42.756638 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.756606 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:10:42.771602 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.771538 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f123c04d-0210-4307-b6c7-1f25de41c2a2-openshift-service-ca\") pod \"f123c04d-0210-4307-b6c7-1f25de41c2a2\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " Jul 08 18:10:42.771744 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.771619 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkj7r\" (UniqueName: \"kubernetes.io/projected/f123c04d-0210-4307-b6c7-1f25de41c2a2-kube-api-access-jkj7r\") pod \"f123c04d-0210-4307-b6c7-1f25de41c2a2\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " Jul 08 18:10:42.771744 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.771720 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls\") pod \"f123c04d-0210-4307-b6c7-1f25de41c2a2\" (UID: \"f123c04d-0210-4307-b6c7-1f25de41c2a2\") " Jul 08 18:10:42.771974 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.771950 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f123c04d-0210-4307-b6c7-1f25de41c2a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f123c04d-0210-4307-b6c7-1f25de41c2a2" (UID: "f123c04d-0210-4307-b6c7-1f25de41c2a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:10:42.773891 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.773862 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f123c04d-0210-4307-b6c7-1f25de41c2a2" (UID: "f123c04d-0210-4307-b6c7-1f25de41c2a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:10:42.773891 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.773870 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f123c04d-0210-4307-b6c7-1f25de41c2a2-kube-api-access-jkj7r" (OuterVolumeSpecName: "kube-api-access-jkj7r") pod "f123c04d-0210-4307-b6c7-1f25de41c2a2" (UID: "f123c04d-0210-4307-b6c7-1f25de41c2a2"). InnerVolumeSpecName "kube-api-access-jkj7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:10:42.872900 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.872809 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f123c04d-0210-4307-b6c7-1f25de41c2a2-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:42.872900 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.872891 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkj7r\" (UniqueName: \"kubernetes.io/projected/f123c04d-0210-4307-b6c7-1f25de41c2a2-kube-api-access-jkj7r\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:42.872900 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:42.872903 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f123c04d-0210-4307-b6c7-1f25de41c2a2-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:10:43.506595 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.506553 2574 generic.go:356] "Generic (PLEG): container finished" podID="f123c04d-0210-4307-b6c7-1f25de41c2a2" containerID="f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a" exitCode=0 Jul 08 18:10:43.506770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.506604 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" Jul 08 18:10:43.506770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.506633 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" event={"ID":"f123c04d-0210-4307-b6c7-1f25de41c2a2","Type":"ContainerDied","Data":"f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a"} Jul 08 18:10:43.506770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.506687 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h" event={"ID":"f123c04d-0210-4307-b6c7-1f25de41c2a2","Type":"ContainerDied","Data":"21e8c5b86751f96601b1ff8806fc3192c273930cfdf89c7643f93170b89e8662"} Jul 08 18:10:43.506770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.506716 2574 scope.go:117] "RemoveContainer" containerID="f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a" Jul 08 18:10:43.516027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.516006 2574 scope.go:117] "RemoveContainer" containerID="f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a" Jul 08 18:10:43.516314 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:10:43.516295 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a\": container with ID starting with f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a not found: ID does not exist" containerID="f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a" Jul 08 18:10:43.516371 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.516323 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a"} err="failed to get container status \"f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a\": rpc error: code = NotFound desc = could not find container \"f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a\": container with ID starting with f8017cb8d6b052c4447169cd47632b3aca8bcf7092c1fcfab3e07ba92224883a not found: ID does not exist" Jul 08 18:10:43.529217 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.529187 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h"] Jul 08 18:10:43.536766 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:43.536741 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c101a9-6454c88d66-zsq8h"] Jul 08 18:10:44.064194 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:44.064158 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f123c04d-0210-4307-b6c7-1f25de41c2a2" path="/var/lib/kubelet/pods/f123c04d-0210-4307-b6c7-1f25de41c2a2/volumes" Jul 08 18:10:49.218296 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218262 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6"] Jul 08 18:10:49.218664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218596 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a71998aa-bdd9-4931-85b1-bcc0d4160f36" containerName="main" Jul 08 18:10:49.218664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218608 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a71998aa-bdd9-4931-85b1-bcc0d4160f36" containerName="main" Jul 08 18:10:49.218664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218631 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b57c094-1fab-45c5-a7a4-090648374589" containerName="maas-api" Jul 08 18:10:49.218664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218637 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b57c094-1fab-45c5-a7a4-090648374589" containerName="maas-api" Jul 08 18:10:49.218664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218643 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f123c04d-0210-4307-b6c7-1f25de41c2a2" containerName="maas-api" Jul 08 18:10:49.218664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218649 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f123c04d-0210-4307-b6c7-1f25de41c2a2" containerName="maas-api" Jul 08 18:10:49.218929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218704 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b57c094-1fab-45c5-a7a4-090648374589" containerName="maas-api" Jul 08 18:10:49.218929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218711 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f123c04d-0210-4307-b6c7-1f25de41c2a2" containerName="maas-api" Jul 08 18:10:49.218929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.218720 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a71998aa-bdd9-4931-85b1-bcc0d4160f36" containerName="main" Jul 08 18:10:49.222949 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.222930 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.225465 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.225445 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b765f0-openshift-default-dockercfg-pb6kw\"" Jul 08 18:10:49.239520 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.237644 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6"] Jul 08 18:10:49.331016 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.330973 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-podinfo\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331016 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331023 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-socket\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331227 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331048 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw78d\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-kube-api-access-mw78d\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331227 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-envoy\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331227 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331182 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-credential-socket\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331227 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331205 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-data\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331372 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-certs\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331372 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9fb48b-d993-4449-b3f8-5880e88a246a-istiod-ca-cert\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.331372 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.331295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-token\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432147 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432104 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-envoy\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432147 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432148 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-credential-socket\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432168 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-data\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-certs\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432247 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9fb48b-d993-4449-b3f8-5880e88a246a-istiod-ca-cert\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-token\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432315 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-podinfo\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-socket\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432406 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432393 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw78d\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-kube-api-access-mw78d\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432756 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432650 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-credential-socket\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432756 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432674 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-data\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432884 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432764 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-certs\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.432884 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.432813 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-socket\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.433136 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.433077 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9fb48b-d993-4449-b3f8-5880e88a246a-istiod-ca-cert\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.434661 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.434640 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-envoy\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.435118 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.435101 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-podinfo\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.441212 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.441178 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-token\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.441314 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.441299 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw78d\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-kube-api-access-mw78d\") pod \"e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.533191 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.533094 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:49.679500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:49.679475 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6"] Jul 08 18:10:49.681416 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:10:49.681389 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9fb48b_d993_4449_b3f8_5880e88a246a.slice/crio-9a09373a4843ad398a6a74f4d8b34ab7ed5d3e7e6ee7a6e07c6c7fb9624a350e WatchSource:0}: Error finding container 9a09373a4843ad398a6a74f4d8b34ab7ed5d3e7e6ee7a6e07c6c7fb9624a350e: Status 404 returned error can't find the container with id 9a09373a4843ad398a6a74f4d8b34ab7ed5d3e7e6ee7a6e07c6c7fb9624a350e Jul 08 18:10:50.545910 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:50.545865 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" event={"ID":"6e9fb48b-d993-4449-b3f8-5880e88a246a","Type":"ContainerStarted","Data":"9a09373a4843ad398a6a74f4d8b34ab7ed5d3e7e6ee7a6e07c6c7fb9624a350e"} Jul 08 18:10:52.143808 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:52.143770 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:10:52.144062 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:52.143867 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:10:52.144062 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:52.143898 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:10:52.557976 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:52.557891 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" event={"ID":"6e9fb48b-d993-4449-b3f8-5880e88a246a","Type":"ContainerStarted","Data":"5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e"} Jul 08 18:10:52.580784 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:52.580730 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podStartSLOduration=1.120556879 podStartE2EDuration="3.580711924s" totalImagesPullingTime="2.460155045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:52.578425019 +0000 UTC m=+2225.078204742" watchObservedRunningTime="2026-07-08 18:10:52.580711924 +0000 UTC m=+2225.080491646" Jul 08 18:10:53.533624 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:53.533589 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:53.538717 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:53.538692 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:53.564060 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:53.564035 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:53.564976 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:53.564961 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:10:54.710243 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.710206 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj"] Jul 08 18:10:54.714328 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.714300 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.716685 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.716657 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b765f0-serving-cert\"" Jul 08 18:10:54.716815 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.716729 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:10:54.717420 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.717400 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fljv6\"" Jul 08 18:10:54.717503 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.717407 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:10:54.725347 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.725320 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj"] Jul 08 18:10:54.779974 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.779937 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.779974 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.779974 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081ba4a1-a7d1-48b8-add7-7478fc1f358e-openshift-service-ca\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.780200 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.780061 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wbk2\" (UniqueName: \"kubernetes.io/projected/081ba4a1-a7d1-48b8-add7-7478fc1f358e-kube-api-access-2wbk2\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.880580 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.880531 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wbk2\" (UniqueName: \"kubernetes.io/projected/081ba4a1-a7d1-48b8-add7-7478fc1f358e-kube-api-access-2wbk2\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.880758 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.880628 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.880758 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.880649 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081ba4a1-a7d1-48b8-add7-7478fc1f358e-openshift-service-ca\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.880863 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:10:54.880778 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-serving-cert: secret "maas-api-e2e-api-a-b765f0-serving-cert" not found Jul 08 18:10:54.880909 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:10:54.880902 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls podName:081ba4a1-a7d1-48b8-add7-7478fc1f358e nodeName:}" failed. No retries permitted until 2026-07-08 18:10:55.380870668 +0000 UTC m=+2227.880650371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls") pod "maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" (UID: "081ba4a1-a7d1-48b8-add7-7478fc1f358e") : secret "maas-api-e2e-api-a-b765f0-serving-cert" not found Jul 08 18:10:54.881346 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.881327 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081ba4a1-a7d1-48b8-add7-7478fc1f358e-openshift-service-ca\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:54.892285 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:54.892256 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wbk2\" (UniqueName: \"kubernetes.io/projected/081ba4a1-a7d1-48b8-add7-7478fc1f358e-kube-api-access-2wbk2\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:55.386532 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:55.386497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:55.389434 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:55.389406 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls\") pod \"maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:55.626622 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:55.626585 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:55.765744 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:55.765716 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj"] Jul 08 18:10:55.767169 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:10:55.767139 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081ba4a1_a7d1_48b8_add7_7478fc1f358e.slice/crio-0145a973c95984663b0dc3714186a6c89c6d23a60482d3b143c22c8848c57bcd WatchSource:0}: Error finding container 0145a973c95984663b0dc3714186a6c89c6d23a60482d3b143c22c8848c57bcd: Status 404 returned error can't find the container with id 0145a973c95984663b0dc3714186a6c89c6d23a60482d3b143c22c8848c57bcd Jul 08 18:10:56.580905 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:56.580862 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" event={"ID":"081ba4a1-a7d1-48b8-add7-7478fc1f358e","Type":"ContainerStarted","Data":"5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85"} Jul 08 18:10:56.581047 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:56.580919 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" event={"ID":"081ba4a1-a7d1-48b8-add7-7478fc1f358e","Type":"ContainerStarted","Data":"0145a973c95984663b0dc3714186a6c89c6d23a60482d3b143c22c8848c57bcd"} Jul 08 18:10:56.581047 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:56.580946 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:10:56.602928 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:10:56.602806 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" podStartSLOduration=2.037895683 podStartE2EDuration="2.602791703s" totalImagesPullingTime="564.89602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:56.601392178 +0000 UTC m=+2229.101171937" watchObservedRunningTime="2026-07-08 18:10:56.602791703 +0000 UTC m=+2229.102571424" Jul 08 18:11:02.605185 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:02.605129 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:11:03.378997 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.378964 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p"] Jul 08 18:11:03.383059 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.383034 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.389058 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.387180 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-78cd4b-serving-cert\"" Jul 08 18:11:03.405170 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.405148 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p"] Jul 08 18:11:03.462605 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.462573 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.462605 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.462607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6zjd\" (UniqueName: \"kubernetes.io/projected/2921d35b-ff9a-4f70-b577-8d759090a076-kube-api-access-r6zjd\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.462790 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.462685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2921d35b-ff9a-4f70-b577-8d759090a076-openshift-service-ca\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.564136 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.564100 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.564136 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.564140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6zjd\" (UniqueName: \"kubernetes.io/projected/2921d35b-ff9a-4f70-b577-8d759090a076-kube-api-access-r6zjd\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.564342 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.564207 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2921d35b-ff9a-4f70-b577-8d759090a076-openshift-service-ca\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.564342 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:03.564255 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-serving-cert: secret "maas-api-e2e-api-b-78cd4b-serving-cert" not found Jul 08 18:11:03.564342 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:03.564329 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls podName:2921d35b-ff9a-4f70-b577-8d759090a076 nodeName:}" failed. No retries permitted until 2026-07-08 18:11:04.064308484 +0000 UTC m=+2236.564088205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls") pod "maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" (UID: "2921d35b-ff9a-4f70-b577-8d759090a076") : secret "maas-api-e2e-api-b-78cd4b-serving-cert" not found Jul 08 18:11:03.564888 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.564870 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2921d35b-ff9a-4f70-b577-8d759090a076-openshift-service-ca\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:03.580396 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:03.580366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6zjd\" (UniqueName: \"kubernetes.io/projected/2921d35b-ff9a-4f70-b577-8d759090a076-kube-api-access-r6zjd\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:04.069762 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:04.069679 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:04.072443 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:04.072415 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls\") pod \"maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:04.304761 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:04.304713 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:04.462113 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:04.462084 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p"] Jul 08 18:11:04.464937 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:11:04.464896 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2921d35b_ff9a_4f70_b577_8d759090a076.slice/crio-11b8ec7c115e68a74b7e61ff34f58a40cd6a716c244bcb953bc8ea62e79f0b45 WatchSource:0}: Error finding container 11b8ec7c115e68a74b7e61ff34f58a40cd6a716c244bcb953bc8ea62e79f0b45: Status 404 returned error can't find the container with id 11b8ec7c115e68a74b7e61ff34f58a40cd6a716c244bcb953bc8ea62e79f0b45 Jul 08 18:11:04.466386 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:04.466350 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:11:04.631048 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:04.630952 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" event={"ID":"2921d35b-ff9a-4f70-b577-8d759090a076","Type":"ContainerStarted","Data":"11b8ec7c115e68a74b7e61ff34f58a40cd6a716c244bcb953bc8ea62e79f0b45"} Jul 08 18:11:05.637592 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:05.637553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" event={"ID":"2921d35b-ff9a-4f70-b577-8d759090a076","Type":"ContainerStarted","Data":"37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c"} Jul 08 18:11:05.638018 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:05.637703 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:05.659308 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:05.659257 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" podStartSLOduration=2.31547796 podStartE2EDuration="2.659243036s" totalImagesPullingTime="343.765076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:05.657361658 +0000 UTC m=+2238.157141424" watchObservedRunningTime="2026-07-08 18:11:05.659243036 +0000 UTC m=+2238.159022760" Jul 08 18:11:06.644182 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:06.644153 2574 generic.go:356] "Generic (PLEG): container finished" podID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerID="5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e" exitCode=0 Jul 08 18:11:06.644579 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:06.644241 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" event={"ID":"6e9fb48b-d993-4449-b3f8-5880e88a246a","Type":"ContainerDied","Data":"5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e"} Jul 08 18:11:06.644736 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:06.644720 2574 scope.go:117] "RemoveContainer" containerID="5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e" Jul 08 18:11:06.646767 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:06.646706 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:06.646868 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:06.646806 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:06.646910 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:06.646871 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:07.652551 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:07.652517 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" event={"ID":"6e9fb48b-d993-4449-b3f8-5880e88a246a","Type":"ContainerStarted","Data":"58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db"} Jul 08 18:11:08.533939 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:08.533904 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:11:08.535142 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:08.535117 2574 patch_prober.go:28] interesting pod/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:11:08.535297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:08.535169 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 18:11:08.564053 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:08.564024 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:11:09.533983 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:09.533947 2574 patch_prober.go:28] interesting pod/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:11:09.534390 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:09.534004 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 18:11:10.533815 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:10.533781 2574 patch_prober.go:28] interesting pod/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:11:10.533994 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:10.533858 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 18:11:11.534280 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:11.534229 2574 patch_prober.go:28] interesting pod/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:11:11.534680 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:11.534324 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 18:11:11.650518 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:11.650485 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:12.537950 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:12.537917 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:11:12.538990 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:12.538970 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:11:16.397015 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.396982 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj"] Jul 08 18:11:16.397471 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.397236 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" podUID="081ba4a1-a7d1-48b8-add7-7478fc1f358e" containerName="maas-api" containerID="cri-o://5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85" gracePeriod=30 Jul 08 18:11:16.667644 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.667582 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:11:16.701741 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.701714 2574 generic.go:356] "Generic (PLEG): container finished" podID="081ba4a1-a7d1-48b8-add7-7478fc1f358e" containerID="5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85" exitCode=0 Jul 08 18:11:16.701889 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.701754 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" Jul 08 18:11:16.701889 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.701792 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" event={"ID":"081ba4a1-a7d1-48b8-add7-7478fc1f358e","Type":"ContainerDied","Data":"5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85"} Jul 08 18:11:16.701889 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.701863 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj" event={"ID":"081ba4a1-a7d1-48b8-add7-7478fc1f358e","Type":"ContainerDied","Data":"0145a973c95984663b0dc3714186a6c89c6d23a60482d3b143c22c8848c57bcd"} Jul 08 18:11:16.701889 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.701890 2574 scope.go:117] "RemoveContainer" containerID="5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85" Jul 08 18:11:16.711022 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.710992 2574 scope.go:117] "RemoveContainer" containerID="5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85" Jul 08 18:11:16.711281 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:16.711260 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85\": container with ID starting with 5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85 not found: ID does not exist" containerID="5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85" Jul 08 18:11:16.711357 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.711297 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85"} err="failed to get container status \"5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85\": rpc error: code = NotFound desc = could not find container \"5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85\": container with ID starting with 5f14dccce4ae04ac986380e068205981c78749ed767c96a517622e5953901f85 not found: ID does not exist" Jul 08 18:11:16.788915 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.788873 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081ba4a1-a7d1-48b8-add7-7478fc1f358e-openshift-service-ca\") pod \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " Jul 08 18:11:16.789105 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.788966 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls\") pod \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " Jul 08 18:11:16.789105 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.789063 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wbk2\" (UniqueName: \"kubernetes.io/projected/081ba4a1-a7d1-48b8-add7-7478fc1f358e-kube-api-access-2wbk2\") pod \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\" (UID: \"081ba4a1-a7d1-48b8-add7-7478fc1f358e\") " Jul 08 18:11:16.789263 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.789238 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081ba4a1-a7d1-48b8-add7-7478fc1f358e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "081ba4a1-a7d1-48b8-add7-7478fc1f358e" (UID: "081ba4a1-a7d1-48b8-add7-7478fc1f358e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:11:16.789364 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.789344 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081ba4a1-a7d1-48b8-add7-7478fc1f358e-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:16.791217 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.791193 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "081ba4a1-a7d1-48b8-add7-7478fc1f358e" (UID: "081ba4a1-a7d1-48b8-add7-7478fc1f358e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:11:16.791310 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.791232 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081ba4a1-a7d1-48b8-add7-7478fc1f358e-kube-api-access-2wbk2" (OuterVolumeSpecName: "kube-api-access-2wbk2") pod "081ba4a1-a7d1-48b8-add7-7478fc1f358e" (UID: "081ba4a1-a7d1-48b8-add7-7478fc1f358e"). InnerVolumeSpecName "kube-api-access-2wbk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:11:16.890853 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.890789 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wbk2\" (UniqueName: \"kubernetes.io/projected/081ba4a1-a7d1-48b8-add7-7478fc1f358e-kube-api-access-2wbk2\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:16.890853 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:16.890852 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081ba4a1-a7d1-48b8-add7-7478fc1f358e-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:17.025387 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:17.025353 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj"] Jul 08 18:11:17.030130 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:17.030106 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b765f0-5d7ffc755f-rfzdj"] Jul 08 18:11:18.064461 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:18.064426 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081ba4a1-a7d1-48b8-add7-7478fc1f358e" path="/var/lib/kubelet/pods/081ba4a1-a7d1-48b8-add7-7478fc1f358e/volumes" Jul 08 18:11:22.471343 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:22.471311 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6"] Jul 08 18:11:22.471723 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:22.471509 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" containerID="cri-o://58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db" gracePeriod=30 Jul 08 18:11:23.756448 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:23.756412 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p"] Jul 08 18:11:23.756919 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:23.756681 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" podUID="2921d35b-ff9a-4f70-b577-8d759090a076" containerName="maas-api" containerID="cri-o://37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c" gracePeriod=30 Jul 08 18:11:24.006033 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.006006 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:24.156487 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.156457 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2921d35b-ff9a-4f70-b577-8d759090a076-openshift-service-ca\") pod \"2921d35b-ff9a-4f70-b577-8d759090a076\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " Jul 08 18:11:24.156653 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.156506 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls\") pod \"2921d35b-ff9a-4f70-b577-8d759090a076\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " Jul 08 18:11:24.156653 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.156529 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6zjd\" (UniqueName: \"kubernetes.io/projected/2921d35b-ff9a-4f70-b577-8d759090a076-kube-api-access-r6zjd\") pod \"2921d35b-ff9a-4f70-b577-8d759090a076\" (UID: \"2921d35b-ff9a-4f70-b577-8d759090a076\") " Jul 08 18:11:24.156897 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.156872 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2921d35b-ff9a-4f70-b577-8d759090a076-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2921d35b-ff9a-4f70-b577-8d759090a076" (UID: "2921d35b-ff9a-4f70-b577-8d759090a076"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:11:24.158858 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.158812 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2921d35b-ff9a-4f70-b577-8d759090a076" (UID: "2921d35b-ff9a-4f70-b577-8d759090a076"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:11:24.158940 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.158903 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2921d35b-ff9a-4f70-b577-8d759090a076-kube-api-access-r6zjd" (OuterVolumeSpecName: "kube-api-access-r6zjd") pod "2921d35b-ff9a-4f70-b577-8d759090a076" (UID: "2921d35b-ff9a-4f70-b577-8d759090a076"). InnerVolumeSpecName "kube-api-access-r6zjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:11:24.257450 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.257415 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2921d35b-ff9a-4f70-b577-8d759090a076-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:24.257450 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.257444 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2921d35b-ff9a-4f70-b577-8d759090a076-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:24.257450 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.257454 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6zjd\" (UniqueName: \"kubernetes.io/projected/2921d35b-ff9a-4f70-b577-8d759090a076-kube-api-access-r6zjd\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:24.747792 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.747752 2574 generic.go:356] "Generic (PLEG): container finished" podID="2921d35b-ff9a-4f70-b577-8d759090a076" containerID="37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c" exitCode=0 Jul 08 18:11:24.748033 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.747804 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" Jul 08 18:11:24.748033 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.747864 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" event={"ID":"2921d35b-ff9a-4f70-b577-8d759090a076","Type":"ContainerDied","Data":"37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c"} Jul 08 18:11:24.748033 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.747920 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p" event={"ID":"2921d35b-ff9a-4f70-b577-8d759090a076","Type":"ContainerDied","Data":"11b8ec7c115e68a74b7e61ff34f58a40cd6a716c244bcb953bc8ea62e79f0b45"} Jul 08 18:11:24.748033 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.747943 2574 scope.go:117] "RemoveContainer" containerID="37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c" Jul 08 18:11:24.757116 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.757099 2574 scope.go:117] "RemoveContainer" containerID="37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c" Jul 08 18:11:24.757423 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:24.757382 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c\": container with ID starting with 37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c not found: ID does not exist" containerID="37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c" Jul 08 18:11:24.757423 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.757407 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c"} err="failed to get container status \"37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c\": rpc error: code = NotFound desc = could not find container \"37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c\": container with ID starting with 37e9d3f36e7fe5f4d307a63289defa5240e3eb7f9b1437c8e3b6946f3fe7e15c not found: ID does not exist" Jul 08 18:11:24.773099 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.773074 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p"] Jul 08 18:11:24.781942 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:24.781919 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-78cd4b-55fdf9d956-gq52p"] Jul 08 18:11:26.064123 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:26.064090 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2921d35b-ff9a-4f70-b577-8d759090a076" path="/var/lib/kubelet/pods/2921d35b-ff9a-4f70-b577-8d759090a076/volumes" Jul 08 18:11:27.540251 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.540217 2574 patch_prober.go:28] interesting pod/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 08 18:11:27.540575 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.540287 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 08 18:11:27.729204 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.729180 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:11:27.765050 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.764976 2574 generic.go:356] "Generic (PLEG): container finished" podID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerID="58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db" exitCode=0 Jul 08 18:11:27.765169 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.765044 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" Jul 08 18:11:27.765169 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.765067 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" event={"ID":"6e9fb48b-d993-4449-b3f8-5880e88a246a","Type":"ContainerDied","Data":"58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db"} Jul 08 18:11:27.765169 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.765127 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6" event={"ID":"6e9fb48b-d993-4449-b3f8-5880e88a246a","Type":"ContainerDied","Data":"9a09373a4843ad398a6a74f4d8b34ab7ed5d3e7e6ee7a6e07c6c7fb9624a350e"} Jul 08 18:11:27.765169 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.765150 2574 scope.go:117] "RemoveContainer" containerID="58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db" Jul 08 18:11:27.773914 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.773896 2574 scope.go:117] "RemoveContainer" containerID="5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e" Jul 08 18:11:27.781671 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.781653 2574 scope.go:117] "RemoveContainer" containerID="58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db" Jul 08 18:11:27.781965 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:27.781944 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db\": container with ID starting with 58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db not found: ID does not exist" containerID="58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db" Jul 08 18:11:27.782023 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.781973 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db"} err="failed to get container status \"58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db\": rpc error: code = NotFound desc = could not find container \"58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db\": container with ID starting with 58e0ca1e1e37dffd73c1db409377458d39f4080657ca59fb3c5a52bd67a020db not found: ID does not exist" Jul 08 18:11:27.782023 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.781991 2574 scope.go:117] "RemoveContainer" containerID="5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e" Jul 08 18:11:27.782115 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.781992 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw78d\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-kube-api-access-mw78d\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782115 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782075 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9fb48b-d993-4449-b3f8-5880e88a246a-istiod-ca-cert\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782209 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782114 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-credential-socket\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782209 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782131 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-data\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782209 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782150 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-token\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782225 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-certs\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782350 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:27.782224 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e\": container with ID starting with 5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e not found: ID does not exist" containerID="5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e" Jul 08 18:11:27.782350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782265 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-socket\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782262 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e"} err="failed to get container status \"5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e\": rpc error: code = NotFound desc = could not find container \"5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e\": container with ID starting with 5300daf555ca4b6e5379a89327dd0819cde9d58749a20d066137e39df35b003e not found: ID does not exist" Jul 08 18:11:27.782350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782328 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-envoy\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782590 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782360 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-podinfo\") pod \"6e9fb48b-d993-4449-b3f8-5880e88a246a\" (UID: \"6e9fb48b-d993-4449-b3f8-5880e88a246a\") " Jul 08 18:11:27.782590 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782388 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:11:27.782590 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782486 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9fb48b-d993-4449-b3f8-5880e88a246a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:11:27.782749 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782696 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:11:27.782749 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782736 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9fb48b-d993-4449-b3f8-5880e88a246a-istiod-ca-cert\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.782877 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782760 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-credential-socket\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.782934 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.782888 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:11:27.783201 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.783181 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:11:27.784719 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.784690 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:11:27.784968 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.784941 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-kube-api-access-mw78d" (OuterVolumeSpecName: "kube-api-access-mw78d") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "kube-api-access-mw78d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:11:27.784968 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.784951 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:11:27.785083 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.785006 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e9fb48b-d993-4449-b3f8-5880e88a246a" (UID: "6e9fb48b-d993-4449-b3f8-5880e88a246a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:11:27.884017 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.883988 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-certs\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.884017 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.884013 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-workload-socket\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.884017 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.884023 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-envoy\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.884197 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.884032 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-podinfo\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.884197 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.884040 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw78d\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-kube-api-access-mw78d\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.884197 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.884048 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-data\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:27.884197 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:27.884055 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9fb48b-d993-4449-b3f8-5880e88a246a-istio-token\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:11:28.088479 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:28.088447 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6"] Jul 08 18:11:28.092245 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:28.092223 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b765f0-openshift-default-664f84d65f-xwdb6"] Jul 08 18:11:30.070529 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.070485 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" path="/var/lib/kubelet/pods/6e9fb48b-d993-4449-b3f8-5880e88a246a/volumes" Jul 08 18:11:30.846878 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.846842 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn"] Jul 08 18:11:30.847242 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847224 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847245 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847255 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847261 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847271 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2921d35b-ff9a-4f70-b577-8d759090a076" containerName="maas-api" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847276 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2921d35b-ff9a-4f70-b577-8d759090a076" containerName="maas-api" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847292 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081ba4a1-a7d1-48b8-add7-7478fc1f358e" containerName="maas-api" Jul 08 18:11:30.847297 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847297 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="081ba4a1-a7d1-48b8-add7-7478fc1f358e" containerName="maas-api" Jul 08 18:11:30.847500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847353 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2921d35b-ff9a-4f70-b577-8d759090a076" containerName="maas-api" Jul 08 18:11:30.847500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847364 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="081ba4a1-a7d1-48b8-add7-7478fc1f358e" containerName="maas-api" Jul 08 18:11:30.847500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847371 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" Jul 08 18:11:30.847500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.847377 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e9fb48b-d993-4449-b3f8-5880e88a246a" containerName="istio-proxy" Jul 08 18:11:30.850875 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.850855 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.853462 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.853436 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-608a65-openshift-default-dockercfg-qtmd9\"" Jul 08 18:11:30.864809 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.864785 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn"] Jul 08 18:11:30.903982 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.903948 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-socket\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904008 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-certs\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904027 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-envoy\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-data\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904064 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6175e263-208d-4c61-85ab-24f67d52a910-istiod-ca-cert\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904153 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-credential-socket\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904348 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904145 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96f47\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-kube-api-access-96f47\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904348 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6175e263-208d-4c61-85ab-24f67d52a910-istio-podinfo\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:30.904348 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:30.904246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-istio-token\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005343 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-socket\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005392 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-certs\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005420 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-envoy\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005441 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-data\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005462 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6175e263-208d-4c61-85ab-24f67d52a910-istiod-ca-cert\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005496 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-credential-socket\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005524 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96f47\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-kube-api-access-96f47\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005570 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6175e263-208d-4c61-85ab-24f67d52a910-istio-podinfo\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.005960 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.005602 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-istio-token\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.006057 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.006031 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-certs\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.006161 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.006142 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-socket\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.006215 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.006183 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-credential-socket\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.006314 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.006290 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6175e263-208d-4c61-85ab-24f67d52a910-istiod-ca-cert\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.006417 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.006396 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-data\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.007951 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.007928 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-envoy\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.008259 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.008239 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6175e263-208d-4c61-85ab-24f67d52a910-istio-podinfo\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.014183 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.014155 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-istio-token\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.014183 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.014165 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96f47\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-kube-api-access-96f47\") pod \"e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.161051 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.160963 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:31.307508 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.307468 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn"] Jul 08 18:11:31.309164 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:11:31.309137 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6175e263_208d_4c61_85ab_24f67d52a910.slice/crio-3803ad23646f79d3d6e0368eb6c6ee4d46fa1004bdba168faa0dfb4a4cffc76e WatchSource:0}: Error finding container 3803ad23646f79d3d6e0368eb6c6ee4d46fa1004bdba168faa0dfb4a4cffc76e: Status 404 returned error can't find the container with id 3803ad23646f79d3d6e0368eb6c6ee4d46fa1004bdba168faa0dfb4a4cffc76e Jul 08 18:11:31.311321 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.311282 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:31.311408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.311353 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:31.311408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.311383 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:31.790680 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.790638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" event={"ID":"6175e263-208d-4c61-85ab-24f67d52a910","Type":"ContainerStarted","Data":"1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba"} Jul 08 18:11:31.790909 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.790707 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" event={"ID":"6175e263-208d-4c61-85ab-24f67d52a910","Type":"ContainerStarted","Data":"3803ad23646f79d3d6e0368eb6c6ee4d46fa1004bdba168faa0dfb4a4cffc76e"} Jul 08 18:11:31.812688 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:31.812636 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podStartSLOduration=1.812615477 podStartE2EDuration="1.812615477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:31.811208955 +0000 UTC m=+2264.310988718" watchObservedRunningTime="2026-07-08 18:11:31.812615477 +0000 UTC m=+2264.312395201" Jul 08 18:11:32.161242 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:32.161210 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:32.166360 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:32.166336 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:32.795595 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:32.795560 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:32.796550 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:32.796533 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:33.853536 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.853504 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v"] Jul 08 18:11:33.856490 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.856470 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:33.859084 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.859056 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-608a65-serving-cert\"" Jul 08 18:11:33.859936 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.859920 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:11:33.860040 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.859919 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:11:33.860040 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.859923 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fljv6\"" Jul 08 18:11:33.873123 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.873094 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v"] Jul 08 18:11:33.932205 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.932169 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxqqp\" (UniqueName: \"kubernetes.io/projected/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-kube-api-access-lxqqp\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:33.932386 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.932223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:33.932386 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:33.932291 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.033157 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.033122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxqqp\" (UniqueName: \"kubernetes.io/projected/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-kube-api-access-lxqqp\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.033342 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.033179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.033342 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.033231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.033432 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:34.033335 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-serving-cert: secret "maas-api-e2e-shared-a-608a65-serving-cert" not found Jul 08 18:11:34.033432 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:34.033393 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls podName:f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1 nodeName:}" failed. No retries permitted until 2026-07-08 18:11:34.533377719 +0000 UTC m=+2267.033157464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls") pod "maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" (UID: "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1") : secret "maas-api-e2e-shared-a-608a65-serving-cert" not found Jul 08 18:11:34.033936 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.033918 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.045274 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.045243 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxqqp\" (UniqueName: \"kubernetes.io/projected/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-kube-api-access-lxqqp\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.538746 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.538709 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.545368 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.542375 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls\") pod \"maas-api-e2e-shared-a-608a65-67557465d6-gpt5v\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.767996 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.767960 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:34.920671 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:34.920642 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v"] Jul 08 18:11:34.923468 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:11:34.923434 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ee43b3_5f80_4e1e_8e41_0f57e73e78c1.slice/crio-227c45ddec2dddabab277267c637a79da2b31314006d287363c10e0decf56c0c WatchSource:0}: Error finding container 227c45ddec2dddabab277267c637a79da2b31314006d287363c10e0decf56c0c: Status 404 returned error can't find the container with id 227c45ddec2dddabab277267c637a79da2b31314006d287363c10e0decf56c0c Jul 08 18:11:35.813210 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:35.813120 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" event={"ID":"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1","Type":"ContainerStarted","Data":"8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30"} Jul 08 18:11:35.813350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:35.813291 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:35.813350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:35.813319 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" event={"ID":"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1","Type":"ContainerStarted","Data":"227c45ddec2dddabab277267c637a79da2b31314006d287363c10e0decf56c0c"} Jul 08 18:11:35.830259 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:35.830204 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" podStartSLOduration=2.258011898 podStartE2EDuration="2.830185386s" totalImagesPullingTime="572.173488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:35.830049656 +0000 UTC m=+2268.329829405" watchObservedRunningTime="2026-07-08 18:11:35.830185386 +0000 UTC m=+2268.329965109" Jul 08 18:11:41.563514 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.563472 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q"] Jul 08 18:11:41.566108 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.566088 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.568930 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.568907 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-63a08f-serving-cert\"" Jul 08 18:11:41.595412 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.595361 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q"] Jul 08 18:11:41.613442 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.613407 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pcxl\" (UniqueName: \"kubernetes.io/projected/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-kube-api-access-5pcxl\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.613642 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.613475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.613642 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.613525 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.714679 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.714645 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pcxl\" (UniqueName: \"kubernetes.io/projected/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-kube-api-access-5pcxl\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.714894 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.714692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.714894 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.714725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.714894 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:41.714878 2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-serving-cert: secret "maas-api-e2e-shared-b-63a08f-serving-cert" not found Jul 08 18:11:41.715084 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:11:41.714963 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls podName:6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb nodeName:}" failed. No retries permitted until 2026-07-08 18:11:42.214940528 +0000 UTC m=+2274.714720231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls") pod "maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" (UID: "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb") : secret "maas-api-e2e-shared-b-63a08f-serving-cert" not found Jul 08 18:11:41.715324 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.715305 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.726409 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.726373 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pcxl\" (UniqueName: \"kubernetes.io/projected/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-kube-api-access-5pcxl\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:41.824770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:41.824741 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:11:42.220873 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:42.220739 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:42.223377 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:42.223356 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls\") pod \"maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:42.479408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:42.479314 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:42.673846 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:42.673789 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q"] Jul 08 18:11:42.676781 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:11:42.676751 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd55cb7_4269_4d58_83a6_c9d1fcc9ebcb.slice/crio-7eb642dce6a31a66b085e4404dc44d3f403490891b803483cf499899deb95819 WatchSource:0}: Error finding container 7eb642dce6a31a66b085e4404dc44d3f403490891b803483cf499899deb95819: Status 404 returned error can't find the container with id 7eb642dce6a31a66b085e4404dc44d3f403490891b803483cf499899deb95819 Jul 08 18:11:42.850090 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:42.850053 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" event={"ID":"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb","Type":"ContainerStarted","Data":"7eb642dce6a31a66b085e4404dc44d3f403490891b803483cf499899deb95819"} Jul 08 18:11:43.856381 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:43.856343 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" event={"ID":"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb","Type":"ContainerStarted","Data":"758ace980599c64bd4b43ba290e45d58dae3145ce6ad984957775b0aa38b0e95"} Jul 08 18:11:43.856887 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:43.856429 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:43.876006 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:43.875958 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" podStartSLOduration=2.419735528 podStartE2EDuration="2.875945631s" totalImagesPullingTime="456.210103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:11:43.873266995 +0000 UTC m=+2276.373046735" watchObservedRunningTime="2026-07-08 18:11:43.875945631 +0000 UTC m=+2276.375725354" Jul 08 18:11:47.797129 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.797089 2574 patch_prober.go:28] interesting pod/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:11:47.797560 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.797159 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 18:11:47.881041 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.881007 2574 generic.go:356] "Generic (PLEG): container finished" podID="6175e263-208d-4c61-85ab-24f67d52a910" containerID="1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba" exitCode=0 Jul 08 18:11:47.881041 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.881049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" event={"ID":"6175e263-208d-4c61-85ab-24f67d52a910","Type":"ContainerDied","Data":"1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba"} Jul 08 18:11:47.881458 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.881443 2574 scope.go:117] "RemoveContainer" containerID="1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba" Jul 08 18:11:47.882523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.882492 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:47.882587 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.882564 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:47.882632 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:47.882597 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 08 18:11:48.893790 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:48.892899 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" event={"ID":"6175e263-208d-4c61-85ab-24f67d52a910","Type":"ContainerStarted","Data":"5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d"} Jul 08 18:11:49.161166 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:49.161066 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:49.161396 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:49.161373 2574 patch_prober.go:28] interesting pod/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:11:49.161442 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:49.161425 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 18:11:49.868469 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:49.868435 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:11:50.162370 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:50.162265 2574 patch_prober.go:28] interesting pod/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:11:50.162370 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:50.162330 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 18:11:51.162090 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:51.162050 2574 patch_prober.go:28] interesting pod/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:11:51.162276 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:51.162110 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 18:11:52.161701 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:52.161663 2574 patch_prober.go:28] interesting pod/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 08 18:11:52.162123 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:52.161722 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 08 18:11:53.165715 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:53.165679 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:53.166105 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:53.165997 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:11:53.166705 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:11:53.166684 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:13:48.141202 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:13:48.141169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:13:48.149651 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:13:48.149630 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:15:00.193221 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.193189 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl"] Jul 08 18:15:00.196101 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.195929 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:15:00.230702 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.230672 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl"] Jul 08 18:15:00.350682 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.350637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5kb5\" (UniqueName: \"kubernetes.io/projected/880f1c44-caa6-43e2-8ffd-571a5d6c4593-kube-api-access-q5kb5\") pod \"maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl\" (UID: \"880f1c44-caa6-43e2-8ffd-571a5d6c4593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:15:00.451666 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.451578 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5kb5\" (UniqueName: \"kubernetes.io/projected/880f1c44-caa6-43e2-8ffd-571a5d6c4593-kube-api-access-q5kb5\") pod \"maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl\" (UID: \"880f1c44-caa6-43e2-8ffd-571a5d6c4593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:15:00.461013 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.460990 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5kb5\" (UniqueName: \"kubernetes.io/projected/880f1c44-caa6-43e2-8ffd-571a5d6c4593-kube-api-access-q5kb5\") pod \"maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl\" (UID: \"880f1c44-caa6-43e2-8ffd-571a5d6c4593\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:15:00.505478 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.505453 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:15:00.652115 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.652088 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl"] Jul 08 18:15:00.653420 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:15:00.653387 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod880f1c44_caa6_43e2_8ffd_571a5d6c4593.slice/crio-bd687562f330199ffa34449242e91c1a266c4bb420ba80a0dcf54d2c248bbb09 WatchSource:0}: Error finding container bd687562f330199ffa34449242e91c1a266c4bb420ba80a0dcf54d2c248bbb09: Status 404 returned error can't find the container with id bd687562f330199ffa34449242e91c1a266c4bb420ba80a0dcf54d2c248bbb09 Jul 08 18:15:00.871538 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:00.871501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" event={"ID":"880f1c44-caa6-43e2-8ffd-571a5d6c4593","Type":"ContainerStarted","Data":"bd687562f330199ffa34449242e91c1a266c4bb420ba80a0dcf54d2c248bbb09"} Jul 08 18:15:01.877763 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:01.877724 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" event={"ID":"880f1c44-caa6-43e2-8ffd-571a5d6c4593","Type":"ContainerStarted","Data":"24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a"} Jul 08 18:15:01.894024 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:15:01.893970 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" podStartSLOduration=1.893952917 podStartE2EDuration="1.893952917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:15:01.892895974 +0000 UTC m=+2474.392675707" watchObservedRunningTime="2026-07-08 18:15:01.893952917 +0000 UTC m=+2474.393732642" Jul 08 18:16:00.012409 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:00.012371 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl"] Jul 08 18:16:00.012869 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:00.012585 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" podUID="880f1c44-caa6-43e2-8ffd-571a5d6c4593" containerName="cleanup" containerID="cri-o://24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a" gracePeriod=30 Jul 08 18:16:12.958925 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.958887 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd"] Jul 08 18:16:12.961664 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.961640 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:12.965720 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.965698 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-608a65\"/\"kube-root-ca.crt\"" Jul 08 18:16:12.966790 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.966578 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-608a65\"/\"openshift-service-ca.crt\"" Jul 08 18:16:12.966790 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.966616 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-608a65\"/\"rate-test-model-608a65-kserve-self-signed-certs\"" Jul 08 18:16:12.967058 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.967034 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-608a65\"/\"default-dockercfg-jrhdx\"" Jul 08 18:16:12.979980 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:12.979955 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd"] Jul 08 18:16:13.054289 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.054253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/951ab109-a211-487e-b30b-c30564b6d9ac-tls-certs\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.054773 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.054311 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-tmp-dir\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.054773 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.054331 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28vdd\" (UniqueName: \"kubernetes.io/projected/951ab109-a211-487e-b30b-c30564b6d9ac-kube-api-access-28vdd\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.054773 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.054357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-home\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.054773 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.054378 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-dshm\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.054773 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.054488 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-model-cache\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.155724 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.155684 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-tmp-dir\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.155952 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.155729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28vdd\" (UniqueName: \"kubernetes.io/projected/951ab109-a211-487e-b30b-c30564b6d9ac-kube-api-access-28vdd\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.155952 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.155771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-home\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.155952 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.155805 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-dshm\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.155952 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.155891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-model-cache\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.155952 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.155944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/951ab109-a211-487e-b30b-c30564b6d9ac-tls-certs\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.156248 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.156227 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-tmp-dir\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.156312 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.156273 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-home\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.156377 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.156336 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-model-cache\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.158287 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.158269 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-dshm\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.158438 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.158421 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/951ab109-a211-487e-b30b-c30564b6d9ac-tls-certs\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.164470 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.164449 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28vdd\" (UniqueName: \"kubernetes.io/projected/951ab109-a211-487e-b30b-c30564b6d9ac-kube-api-access-28vdd\") pod \"rate-test-model-608a65-kserve-77bb45654c-fzsdd\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.274325 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.274241 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:13.423895 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:13.423862 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd"] Jul 08 18:16:13.424923 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:16:13.424888 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod951ab109_a211_487e_b30b_c30564b6d9ac.slice/crio-af6c64f807aae08f9be2ccd6a9b6d2b2098d5a4ef24a68f110ef0015d7805fd3 WatchSource:0}: Error finding container af6c64f807aae08f9be2ccd6a9b6d2b2098d5a4ef24a68f110ef0015d7805fd3: Status 404 returned error can't find the container with id af6c64f807aae08f9be2ccd6a9b6d2b2098d5a4ef24a68f110ef0015d7805fd3 Jul 08 18:16:14.242892 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:14.242856 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" event={"ID":"951ab109-a211-487e-b30b-c30564b6d9ac","Type":"ContainerStarted","Data":"d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49"} Jul 08 18:16:14.243283 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:14.242902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" event={"ID":"951ab109-a211-487e-b30b-c30564b6d9ac","Type":"ContainerStarted","Data":"af6c64f807aae08f9be2ccd6a9b6d2b2098d5a4ef24a68f110ef0015d7805fd3"} Jul 08 18:16:14.260891 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:14.260805 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" podStartSLOduration=2.260786218 podStartE2EDuration="2.260786218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:16:14.260439251 +0000 UTC m=+2546.760218975" watchObservedRunningTime="2026-07-08 18:16:14.260786218 +0000 UTC m=+2546.760565942" Jul 08 18:16:23.274804 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:23.274764 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:23.275308 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:23.274854 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:23.287260 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:23.287236 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:23.302749 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:23.302726 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:16:30.187129 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.187097 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:16:30.302849 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.302738 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5kb5\" (UniqueName: \"kubernetes.io/projected/880f1c44-caa6-43e2-8ffd-571a5d6c4593-kube-api-access-q5kb5\") pod \"880f1c44-caa6-43e2-8ffd-571a5d6c4593\" (UID: \"880f1c44-caa6-43e2-8ffd-571a5d6c4593\") " Jul 08 18:16:30.305015 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.304981 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/880f1c44-caa6-43e2-8ffd-571a5d6c4593-kube-api-access-q5kb5" (OuterVolumeSpecName: "kube-api-access-q5kb5") pod "880f1c44-caa6-43e2-8ffd-571a5d6c4593" (UID: "880f1c44-caa6-43e2-8ffd-571a5d6c4593"). InnerVolumeSpecName "kube-api-access-q5kb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:16:30.332547 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.332511 2574 generic.go:356] "Generic (PLEG): container finished" podID="880f1c44-caa6-43e2-8ffd-571a5d6c4593" containerID="24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a" exitCode=137 Jul 08 18:16:30.332662 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.332555 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" Jul 08 18:16:30.332662 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.332604 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" event={"ID":"880f1c44-caa6-43e2-8ffd-571a5d6c4593","Type":"ContainerDied","Data":"24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a"} Jul 08 18:16:30.332662 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.332659 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl" event={"ID":"880f1c44-caa6-43e2-8ffd-571a5d6c4593","Type":"ContainerDied","Data":"bd687562f330199ffa34449242e91c1a266c4bb420ba80a0dcf54d2c248bbb09"} Jul 08 18:16:30.332767 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.332682 2574 scope.go:117] "RemoveContainer" containerID="24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a" Jul 08 18:16:30.341974 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.341957 2574 scope.go:117] "RemoveContainer" containerID="24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a" Jul 08 18:16:30.342290 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:16:30.342271 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a\": container with ID starting with 24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a not found: ID does not exist" containerID="24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a" Jul 08 18:16:30.342369 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.342300 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a"} err="failed to get container status \"24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a\": rpc error: code = NotFound desc = could not find container \"24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a\": container with ID starting with 24cd09b99b4270c34778c4bab900c0b359ce53025abbaca1b12639ce66bc9e8a not found: ID does not exist" Jul 08 18:16:30.353697 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.353669 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl"] Jul 08 18:16:30.357759 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.357733 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-63a08f-29725575-q8xdl"] Jul 08 18:16:30.404139 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:30.404109 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5kb5\" (UniqueName: \"kubernetes.io/projected/880f1c44-caa6-43e2-8ffd-571a5d6c4593-kube-api-access-q5kb5\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:16:32.064282 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:16:32.064242 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="880f1c44-caa6-43e2-8ffd-571a5d6c4593" path="/var/lib/kubelet/pods/880f1c44-caa6-43e2-8ffd-571a5d6c4593/volumes" Jul 08 18:17:25.339853 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:25.339792 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd"] Jul 08 18:17:40.360663 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.360605 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" podUID="951ab109-a211-487e-b30b-c30564b6d9ac" containerName="main" containerID="cri-o://d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49" gracePeriod=45 Jul 08 18:17:40.619751 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.619681 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:17:40.649326 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649280 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28vdd\" (UniqueName: \"kubernetes.io/projected/951ab109-a211-487e-b30b-c30564b6d9ac-kube-api-access-28vdd\") pod \"951ab109-a211-487e-b30b-c30564b6d9ac\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " Jul 08 18:17:40.649326 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649328 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/951ab109-a211-487e-b30b-c30564b6d9ac-tls-certs\") pod \"951ab109-a211-487e-b30b-c30564b6d9ac\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " Jul 08 18:17:40.649587 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649351 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-model-cache\") pod \"951ab109-a211-487e-b30b-c30564b6d9ac\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " Jul 08 18:17:40.649587 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649382 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-home\") pod \"951ab109-a211-487e-b30b-c30564b6d9ac\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " Jul 08 18:17:40.649587 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649415 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-tmp-dir\") pod \"951ab109-a211-487e-b30b-c30564b6d9ac\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " Jul 08 18:17:40.649587 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649464 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-dshm\") pod \"951ab109-a211-487e-b30b-c30564b6d9ac\" (UID: \"951ab109-a211-487e-b30b-c30564b6d9ac\") " Jul 08 18:17:40.650481 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.650053 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-home" (OuterVolumeSpecName: "home") pod "951ab109-a211-487e-b30b-c30564b6d9ac" (UID: "951ab109-a211-487e-b30b-c30564b6d9ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:17:40.650481 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.650300 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "951ab109-a211-487e-b30b-c30564b6d9ac" (UID: "951ab109-a211-487e-b30b-c30564b6d9ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:17:40.650481 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.649616 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "951ab109-a211-487e-b30b-c30564b6d9ac" (UID: "951ab109-a211-487e-b30b-c30564b6d9ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:17:40.652633 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.652543 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-dshm" (OuterVolumeSpecName: "dshm") pod "951ab109-a211-487e-b30b-c30564b6d9ac" (UID: "951ab109-a211-487e-b30b-c30564b6d9ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:17:40.652633 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.652605 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951ab109-a211-487e-b30b-c30564b6d9ac-kube-api-access-28vdd" (OuterVolumeSpecName: "kube-api-access-28vdd") pod "951ab109-a211-487e-b30b-c30564b6d9ac" (UID: "951ab109-a211-487e-b30b-c30564b6d9ac"). InnerVolumeSpecName "kube-api-access-28vdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:17:40.652813 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.652634 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951ab109-a211-487e-b30b-c30564b6d9ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "951ab109-a211-487e-b30b-c30564b6d9ac" (UID: "951ab109-a211-487e-b30b-c30564b6d9ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:17:40.695867 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.695813 2574 generic.go:356] "Generic (PLEG): container finished" podID="951ab109-a211-487e-b30b-c30564b6d9ac" containerID="d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49" exitCode=0 Jul 08 18:17:40.695867 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.695872 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" Jul 08 18:17:40.696121 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.695895 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" event={"ID":"951ab109-a211-487e-b30b-c30564b6d9ac","Type":"ContainerDied","Data":"d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49"} Jul 08 18:17:40.696121 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.695944 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd" event={"ID":"951ab109-a211-487e-b30b-c30564b6d9ac","Type":"ContainerDied","Data":"af6c64f807aae08f9be2ccd6a9b6d2b2098d5a4ef24a68f110ef0015d7805fd3"} Jul 08 18:17:40.696121 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.695962 2574 scope.go:117] "RemoveContainer" containerID="d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49" Jul 08 18:17:40.705308 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.705283 2574 scope.go:117] "RemoveContainer" containerID="d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49" Jul 08 18:17:40.705567 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:17:40.705547 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49\": container with ID starting with d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49 not found: ID does not exist" containerID="d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49" Jul 08 18:17:40.705622 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.705576 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49"} err="failed to get container status \"d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49\": rpc error: code = NotFound desc = could not find container \"d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49\": container with ID starting with d32f19aa0b5af13d9d73ccd892a56ec9bcfd3cdb1e772a03f7227336d3ccfa49 not found: ID does not exist" Jul 08 18:17:40.723093 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.719731 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd"] Jul 08 18:17:40.727058 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.727022 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-608a65/rate-test-model-608a65-kserve-77bb45654c-fzsdd"] Jul 08 18:17:40.751408 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.751376 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-dshm\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:17:40.751523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.751426 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28vdd\" (UniqueName: \"kubernetes.io/projected/951ab109-a211-487e-b30b-c30564b6d9ac-kube-api-access-28vdd\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:17:40.751523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.751437 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/951ab109-a211-487e-b30b-c30564b6d9ac-tls-certs\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:17:40.751523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.751447 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-model-cache\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:17:40.751523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.751455 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-home\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:17:40.751523 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:40.751462 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/951ab109-a211-487e-b30b-c30564b6d9ac-tmp-dir\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:17:42.064177 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:17:42.064140 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="951ab109-a211-487e-b30b-c30564b6d9ac" path="/var/lib/kubelet/pods/951ab109-a211-487e-b30b-c30564b6d9ac/volumes" Jul 08 18:18:07.902001 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:07.901956 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v"] Jul 08 18:18:07.902611 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:07.902247 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" podUID="f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" containerName="maas-api" containerID="cri-o://8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30" gracePeriod=30 Jul 08 18:18:08.161514 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.161449 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:18:08.305641 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.305606 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-openshift-service-ca\") pod \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " Jul 08 18:18:08.305885 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.305687 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls\") pod \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " Jul 08 18:18:08.305885 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.305705 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxqqp\" (UniqueName: \"kubernetes.io/projected/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-kube-api-access-lxqqp\") pod \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\" (UID: \"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1\") " Jul 08 18:18:08.306076 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.306051 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" (UID: "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:18:08.308082 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.308055 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" (UID: "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:18:08.308179 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.308103 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-kube-api-access-lxqqp" (OuterVolumeSpecName: "kube-api-access-lxqqp") pod "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" (UID: "f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1"). InnerVolumeSpecName "kube-api-access-lxqqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:18:08.407027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.406991 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxqqp\" (UniqueName: \"kubernetes.io/projected/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-kube-api-access-lxqqp\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:08.407027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.407022 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:08.407027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.407035 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:08.853708 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.853669 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" containerID="8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30" exitCode=0 Jul 08 18:18:08.853947 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.853723 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" Jul 08 18:18:08.853947 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.853752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" event={"ID":"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1","Type":"ContainerDied","Data":"8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30"} Jul 08 18:18:08.853947 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.853815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v" event={"ID":"f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1","Type":"ContainerDied","Data":"227c45ddec2dddabab277267c637a79da2b31314006d287363c10e0decf56c0c"} Jul 08 18:18:08.853947 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.853856 2574 scope.go:117] "RemoveContainer" containerID="8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30" Jul 08 18:18:08.863471 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.863451 2574 scope.go:117] "RemoveContainer" containerID="8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30" Jul 08 18:18:08.863730 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:18:08.863710 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30\": container with ID starting with 8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30 not found: ID does not exist" containerID="8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30" Jul 08 18:18:08.863780 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.863739 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30"} err="failed to get container status \"8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30\": rpc error: code = NotFound desc = could not find container \"8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30\": container with ID starting with 8145ad84b1ba2a160996e03cc799c0e698be4173239ccb89cdb8e1f6ed4f4f30 not found: ID does not exist" Jul 08 18:18:08.876256 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.876226 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v"] Jul 08 18:18:08.882757 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:08.882732 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-608a65-67557465d6-gpt5v"] Jul 08 18:18:10.063773 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:10.063724 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" path="/var/lib/kubelet/pods/f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1/volumes" Jul 08 18:18:14.489784 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:14.489748 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn"] Jul 08 18:18:14.490226 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:14.489972 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" containerID="cri-o://5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d" gracePeriod=30 Jul 08 18:18:15.748414 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:15.748379 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q"] Jul 08 18:18:15.748799 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:15.748594 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" podUID="6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" containerName="maas-api" containerID="cri-o://758ace980599c64bd4b43ba290e45d58dae3145ce6ad984957775b0aa38b0e95" gracePeriod=30 Jul 08 18:18:15.893443 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:15.893411 2574 generic.go:356] "Generic (PLEG): container finished" podID="6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" containerID="758ace980599c64bd4b43ba290e45d58dae3145ce6ad984957775b0aa38b0e95" exitCode=0 Jul 08 18:18:15.893579 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:15.893479 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" event={"ID":"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb","Type":"ContainerDied","Data":"758ace980599c64bd4b43ba290e45d58dae3145ce6ad984957775b0aa38b0e95"} Jul 08 18:18:15.993519 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:15.993490 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:18:16.077614 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.077587 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls\") pod \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " Jul 08 18:18:16.077770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.077663 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-openshift-service-ca\") pod \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " Jul 08 18:18:16.077770 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.077717 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pcxl\" (UniqueName: \"kubernetes.io/projected/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-kube-api-access-5pcxl\") pod \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\" (UID: \"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb\") " Jul 08 18:18:16.078091 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.078060 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" (UID: "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:18:16.079739 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.079716 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" (UID: "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:18:16.079887 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.079866 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-kube-api-access-5pcxl" (OuterVolumeSpecName: "kube-api-access-5pcxl") pod "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" (UID: "6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb"). InnerVolumeSpecName "kube-api-access-5pcxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:18:16.178337 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.178303 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-maas-api-tls\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:16.178337 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.178331 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-openshift-service-ca\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:16.178337 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.178343 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pcxl\" (UniqueName: \"kubernetes.io/projected/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb-kube-api-access-5pcxl\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:16.900046 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.900013 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" Jul 08 18:18:16.900463 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.900022 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q" event={"ID":"6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb","Type":"ContainerDied","Data":"7eb642dce6a31a66b085e4404dc44d3f403490891b803483cf499899deb95819"} Jul 08 18:18:16.900463 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.900147 2574 scope.go:117] "RemoveContainer" containerID="758ace980599c64bd4b43ba290e45d58dae3145ce6ad984957775b0aa38b0e95" Jul 08 18:18:16.921777 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.921751 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q"] Jul 08 18:18:16.927404 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:16.927379 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-63a08f-789bb6f8ff-n525q"] Jul 08 18:18:18.065012 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:18.064977 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" path="/var/lib/kubelet/pods/6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb/volumes" Jul 08 18:18:19.742460 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.742437 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:18:19.809300 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809232 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-socket\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809300 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809264 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-certs\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809300 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809286 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-credential-socket\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809525 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809311 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6175e263-208d-4c61-85ab-24f67d52a910-istio-podinfo\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809525 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809342 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-istio-token\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809525 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809391 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-envoy\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809525 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809455 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-data\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809525 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809479 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6175e263-208d-4c61-85ab-24f67d52a910-istiod-ca-cert\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809525 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809508 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96f47\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-kube-api-access-96f47\") pod \"6175e263-208d-4c61-85ab-24f67d52a910\" (UID: \"6175e263-208d-4c61-85ab-24f67d52a910\") " Jul 08 18:18:19.809841 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809550 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:18:19.809841 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809566 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:18:19.809841 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809579 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:18:19.810015 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809868 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-socket\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.810015 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809891 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6175e263-208d-4c61-85ab-24f67d52a910-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:18:19.810015 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809897 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-workload-certs\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.810015 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.809940 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-credential-socket\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.810382 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.810171 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-data" (OuterVolumeSpecName: "istio-data") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:18:19.811807 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.811784 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6175e263-208d-4c61-85ab-24f67d52a910-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 18:18:19.811953 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.811862 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-kube-api-access-96f47" (OuterVolumeSpecName: "kube-api-access-96f47") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "kube-api-access-96f47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:18:19.812243 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.812221 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-istio-token" (OuterVolumeSpecName: "istio-token") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:18:19.812243 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.812224 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6175e263-208d-4c61-85ab-24f67d52a910" (UID: "6175e263-208d-4c61-85ab-24f67d52a910"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:18:19.910835 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.910802 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-envoy\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.910966 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.910846 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6175e263-208d-4c61-85ab-24f67d52a910-istio-data\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.910966 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.910856 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6175e263-208d-4c61-85ab-24f67d52a910-istiod-ca-cert\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.910966 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.910866 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96f47\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-kube-api-access-96f47\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.910966 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.910874 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6175e263-208d-4c61-85ab-24f67d52a910-istio-podinfo\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.910966 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.910883 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6175e263-208d-4c61-85ab-24f67d52a910-istio-token\") on node \"ip-10-0-128-115.ec2.internal\" DevicePath \"\"" Jul 08 18:18:19.930025 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.929991 2574 generic.go:356] "Generic (PLEG): container finished" podID="6175e263-208d-4c61-85ab-24f67d52a910" containerID="5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d" exitCode=0 Jul 08 18:18:19.930157 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.930046 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" Jul 08 18:18:19.930157 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.930081 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" event={"ID":"6175e263-208d-4c61-85ab-24f67d52a910","Type":"ContainerDied","Data":"5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d"} Jul 08 18:18:19.930157 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.930150 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn" event={"ID":"6175e263-208d-4c61-85ab-24f67d52a910","Type":"ContainerDied","Data":"3803ad23646f79d3d6e0368eb6c6ee4d46fa1004bdba168faa0dfb4a4cffc76e"} Jul 08 18:18:19.930309 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.930167 2574 scope.go:117] "RemoveContainer" containerID="5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d" Jul 08 18:18:19.939503 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.939488 2574 scope.go:117] "RemoveContainer" containerID="1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba" Jul 08 18:18:19.947175 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.947154 2574 scope.go:117] "RemoveContainer" containerID="5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d" Jul 08 18:18:19.947464 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:18:19.947442 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d\": container with ID starting with 5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d not found: ID does not exist" containerID="5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d" Jul 08 18:18:19.947511 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.947471 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d"} err="failed to get container status \"5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d\": rpc error: code = NotFound desc = could not find container \"5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d\": container with ID starting with 5b6e9c3607fa791e36b67ac4d861d1c99f0136f802182e791998b28ad6d4756d not found: ID does not exist" Jul 08 18:18:19.947511 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.947491 2574 scope.go:117] "RemoveContainer" containerID="1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba" Jul 08 18:18:19.947754 ip-10-0-128-115 kubenswrapper[2574]: E0708 18:18:19.947737 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba\": container with ID starting with 1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba not found: ID does not exist" containerID="1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba" Jul 08 18:18:19.947795 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.947758 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba"} err="failed to get container status \"1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba\": rpc error: code = NotFound desc = could not find container \"1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba\": container with ID starting with 1ded7335820b2b8d3709eff6752535f216b9d54828ff2e86884377cc2c373bba not found: ID does not exist" Jul 08 18:18:19.951739 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.951714 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn"] Jul 08 18:18:19.955466 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:19.955435 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-608a65-openshift-default-7dff96764c-5vsvn"] Jul 08 18:18:20.065174 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:20.065098 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6175e263-208d-4c61-85ab-24f67d52a910" path="/var/lib/kubelet/pods/6175e263-208d-4c61-85ab-24f67d52a910/volumes" Jul 08 18:18:27.768599 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:27.768563 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c548d5447-9s8jh_483dba4c-3621-4126-ae6f-738ee2d29df1/manager/0.log" Jul 08 18:18:27.776062 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:27.776039 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c548d5447-9s8jh_483dba4c-3621-4126-ae6f-738ee2d29df1/copy-manifests/0.log" Jul 08 18:18:28.568926 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.568897 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4_073f055d-0cef-4283-ab91-ae76b15e1757/util/0.log" Jul 08 18:18:28.576270 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.576245 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4_073f055d-0cef-4283-ab91-ae76b15e1757/pull/0.log" Jul 08 18:18:28.586004 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.585983 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4_073f055d-0cef-4283-ab91-ae76b15e1757/extract/0.log" Jul 08 18:18:28.706400 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.706369 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6_212d8976-41d7-43e0-89d8-cf110169e2db/util/0.log" Jul 08 18:18:28.713284 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.713235 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6_212d8976-41d7-43e0-89d8-cf110169e2db/pull/0.log" Jul 08 18:18:28.719957 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.719937 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6_212d8976-41d7-43e0-89d8-cf110169e2db/extract/0.log" Jul 08 18:18:28.833799 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.833778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk_248e88ad-4ac3-472a-8a0b-1e152e9653e5/util/0.log" Jul 08 18:18:28.840811 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.840795 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk_248e88ad-4ac3-472a-8a0b-1e152e9653e5/pull/0.log" Jul 08 18:18:28.847991 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.847974 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk_248e88ad-4ac3-472a-8a0b-1e152e9653e5/extract/0.log" Jul 08 18:18:28.960274 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.960248 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn_c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9/util/0.log" Jul 08 18:18:28.966481 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.966461 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn_c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9/pull/0.log" Jul 08 18:18:28.973488 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:28.973469 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn_c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9/extract/0.log" Jul 08 18:18:29.949592 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:29.949563 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-krckj_aed853be-8853-4140-ae7f-e947e45c4fff/manager/0.log" Jul 08 18:18:30.444953 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:30.444925 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7brb2_2a66b758-ef3f-4e43-81d2-0d4807741157/discovery/0.log" Jul 08 18:18:30.909996 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:30.909969 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gfqvm_f0abacd3-e2e7-4f66-a201-3bcf597e469b/payload-pre-processing/0.log" Jul 08 18:18:31.031932 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:31.031901 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dpdct_49d6102f-58cd-4cc4-befc-a8ddeb457b5f/payload-processing/0.log" Jul 08 18:18:31.491727 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:31.491699 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wmw7z_06b287b9-2baf-43a9-99fc-0d3c6575f52f/main/0.log" Jul 08 18:18:36.637867 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.637818 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cq44k/must-gather-v5bkf"] Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638168 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="951ab109-a211-487e-b30b-c30564b6d9ac" containerName="main" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638180 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="951ab109-a211-487e-b30b-c30564b6d9ac" containerName="main" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638192 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" containerName="maas-api" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638197 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" containerName="maas-api" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638205 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638211 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638218 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" containerName="maas-api" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638223 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" containerName="maas-api" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638232 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" Jul 08 18:18:36.638234 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638237 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638249 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="880f1c44-caa6-43e2-8ffd-571a5d6c4593" containerName="cleanup" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638254 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="880f1c44-caa6-43e2-8ffd-571a5d6c4593" containerName="cleanup" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638301 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="951ab109-a211-487e-b30b-c30564b6d9ac" containerName="main" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638309 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638316 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6175e263-208d-4c61-85ab-24f67d52a910" containerName="istio-proxy" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638322 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="880f1c44-caa6-43e2-8ffd-571a5d6c4593" containerName="cleanup" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638329 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2ee43b3-5f80-4e1e-8e41-0f57e73e78c1" containerName="maas-api" Jul 08 18:18:36.638544 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.638335 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cd55cb7-4269-4d58-83a6-c9d1fcc9ebcb" containerName="maas-api" Jul 08 18:18:36.641392 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.641364 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.643959 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.643937 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cq44k\"/\"openshift-service-ca.crt\"" Jul 08 18:18:36.644085 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.643944 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cq44k\"/\"kube-root-ca.crt\"" Jul 08 18:18:36.644577 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.644561 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cq44k\"/\"default-dockercfg-5jf2t\"" Jul 08 18:18:36.651141 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.651119 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq44k/must-gather-v5bkf"] Jul 08 18:18:36.748964 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.748935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42-must-gather-output\") pod \"must-gather-v5bkf\" (UID: \"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42\") " pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.749119 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.748992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9qz\" (UniqueName: \"kubernetes.io/projected/8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42-kube-api-access-9h9qz\") pod \"must-gather-v5bkf\" (UID: \"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42\") " pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.849490 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.849447 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42-must-gather-output\") pod \"must-gather-v5bkf\" (UID: \"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42\") " pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.849675 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.849513 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9qz\" (UniqueName: \"kubernetes.io/projected/8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42-kube-api-access-9h9qz\") pod \"must-gather-v5bkf\" (UID: \"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42\") " pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.849855 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.849801 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42-must-gather-output\") pod \"must-gather-v5bkf\" (UID: \"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42\") " pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.857482 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.857459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9qz\" (UniqueName: \"kubernetes.io/projected/8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42-kube-api-access-9h9qz\") pod \"must-gather-v5bkf\" (UID: \"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42\") " pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:36.950786 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:36.950686 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq44k/must-gather-v5bkf" Jul 08 18:18:37.084336 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:37.084297 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq44k/must-gather-v5bkf"] Jul 08 18:18:37.091008 ip-10-0-128-115 kubenswrapper[2574]: W0708 18:18:37.090979 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bbd07dd_9d8d_47c4_a0ad_9d5a3ffa7b42.slice/crio-833c87969ee92a375a53b8004334c022f0cba8a824659ff59e55828f369f2950 WatchSource:0}: Error finding container 833c87969ee92a375a53b8004334c022f0cba8a824659ff59e55828f369f2950: Status 404 returned error can't find the container with id 833c87969ee92a375a53b8004334c022f0cba8a824659ff59e55828f369f2950 Jul 08 18:18:37.092917 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:37.092898 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:18:38.033270 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:38.033165 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq44k/must-gather-v5bkf" event={"ID":"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42","Type":"ContainerStarted","Data":"fc18490a940f1115df8fc5dbfe2ae79f32f455130facf179be124531bbb09b1a"} Jul 08 18:18:38.033270 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:38.033225 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq44k/must-gather-v5bkf" event={"ID":"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42","Type":"ContainerStarted","Data":"a8232a3fe376f3f16de5b5b3beba0652b6f14a59bd288a5d254a4963e03ec8c0"} Jul 08 18:18:38.033270 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:38.033250 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq44k/must-gather-v5bkf" event={"ID":"8bbd07dd-9d8d-47c4-a0ad-9d5a3ffa7b42","Type":"ContainerStarted","Data":"833c87969ee92a375a53b8004334c022f0cba8a824659ff59e55828f369f2950"} Jul 08 18:18:38.049908 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:38.049849 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cq44k/must-gather-v5bkf" podStartSLOduration=1.36668003 podStartE2EDuration="2.049816908s" totalImagesPullingTime="683.136878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:38.047011474 +0000 UTC m=+2690.546791197" watchObservedRunningTime="2026-07-08 18:18:38.049816908 +0000 UTC m=+2690.549596631" Jul 08 18:18:39.467811 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:39.467777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sgg9w_42a39394-26a1-4f87-8f51-709a58922414/global-pull-secret-syncer/0.log" Jul 08 18:18:39.571818 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:39.571783 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rk9qs_af2773a3-d6ec-4fd1-8bf1-81440f6c11cb/konnectivity-agent/0.log" Jul 08 18:18:39.628520 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:39.628492 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-115.ec2.internal_57192b8ecd12bde2baff3a89b0094a6a/haproxy/0.log" Jul 08 18:18:43.563877 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.563850 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4_073f055d-0cef-4283-ab91-ae76b15e1757/extract/0.log" Jul 08 18:18:43.589321 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.589297 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4_073f055d-0cef-4283-ab91-ae76b15e1757/util/0.log" Jul 08 18:18:43.612554 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.612510 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jpnf4_073f055d-0cef-4283-ab91-ae76b15e1757/pull/0.log" Jul 08 18:18:43.651475 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.651423 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6_212d8976-41d7-43e0-89d8-cf110169e2db/extract/0.log" Jul 08 18:18:43.679998 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.679967 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6_212d8976-41d7-43e0-89d8-cf110169e2db/util/0.log" Jul 08 18:18:43.703941 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.703899 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lnwc6_212d8976-41d7-43e0-89d8-cf110169e2db/pull/0.log" Jul 08 18:18:43.732601 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.732543 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk_248e88ad-4ac3-472a-8a0b-1e152e9653e5/extract/0.log" Jul 08 18:18:43.754854 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.754810 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk_248e88ad-4ac3-472a-8a0b-1e152e9653e5/util/0.log" Jul 08 18:18:43.777735 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.777705 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wvftk_248e88ad-4ac3-472a-8a0b-1e152e9653e5/pull/0.log" Jul 08 18:18:43.806528 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.806503 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn_c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9/extract/0.log" Jul 08 18:18:43.844128 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.844089 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn_c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9/util/0.log" Jul 08 18:18:43.865197 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:43.865159 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xzjdn_c2a7f1fa-cbbc-4fcd-8e64-e4f8e91b77a9/pull/0.log" Jul 08 18:18:44.603105 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:44.602152 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-krckj_aed853be-8853-4140-ae7f-e947e45c4fff/manager/0.log" Jul 08 18:18:46.718069 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:46.717962 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-7lw7k_ac4bc77c-7758-4320-9947-3ba2a4cae049/kube-state-metrics/0.log" Jul 08 18:18:46.745579 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:46.745549 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-7lw7k_ac4bc77c-7758-4320-9947-3ba2a4cae049/kube-rbac-proxy-main/0.log" Jul 08 18:18:46.771260 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:46.771212 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-7lw7k_ac4bc77c-7758-4320-9947-3ba2a4cae049/kube-rbac-proxy-self/0.log" Jul 08 18:18:46.807467 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:46.807402 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-cc46778b7-j2cw4_71be1e72-bd7e-4498-989b-bfd5f7328886/metrics-server/0.log" Jul 08 18:18:47.134084 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.134052 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdvfv_e0026793-f06e-44cb-8d51-6aa993a0bd83/node-exporter/0.log" Jul 08 18:18:47.216209 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.216174 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdvfv_e0026793-f06e-44cb-8d51-6aa993a0bd83/kube-rbac-proxy/0.log" Jul 08 18:18:47.338863 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.338806 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gdvfv_e0026793-f06e-44cb-8d51-6aa993a0bd83/init-textfile/0.log" Jul 08 18:18:47.421705 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.421626 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pd4nw_6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9/kube-rbac-proxy-main/0.log" Jul 08 18:18:47.448324 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.448295 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pd4nw_6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9/kube-rbac-proxy-self/0.log" Jul 08 18:18:47.479027 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.478991 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pd4nw_6ead4133-1c89-4d01-b1b4-52a4f7a1b5e9/openshift-state-metrics/0.log" Jul 08 18:18:47.781929 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.781809 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74fd9444fc-lmxzd_db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44/telemeter-client/0.log" Jul 08 18:18:47.806709 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.806674 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74fd9444fc-lmxzd_db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44/reload/0.log" Jul 08 18:18:47.831384 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.831344 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74fd9444fc-lmxzd_db20c1c4-ef59-4ff2-b9fb-1f2c0ced9d44/kube-rbac-proxy/0.log" Jul 08 18:18:47.891199 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.891162 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7df7f97cdf-x55bh_2b56df8b-1785-4365-9e5e-97f59fa3a4cf/thanos-query/0.log" Jul 08 18:18:47.917807 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.917775 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7df7f97cdf-x55bh_2b56df8b-1785-4365-9e5e-97f59fa3a4cf/kube-rbac-proxy-web/0.log" Jul 08 18:18:47.940663 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.940622 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7df7f97cdf-x55bh_2b56df8b-1785-4365-9e5e-97f59fa3a4cf/kube-rbac-proxy/0.log" Jul 08 18:18:47.964950 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.964921 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7df7f97cdf-x55bh_2b56df8b-1785-4365-9e5e-97f59fa3a4cf/prom-label-proxy/0.log" Jul 08 18:18:47.988357 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:47.988312 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7df7f97cdf-x55bh_2b56df8b-1785-4365-9e5e-97f59fa3a4cf/kube-rbac-proxy-rules/0.log" Jul 08 18:18:48.019752 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:48.019712 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7df7f97cdf-x55bh_2b56df8b-1785-4365-9e5e-97f59fa3a4cf/kube-rbac-proxy-metrics/0.log" Jul 08 18:18:48.194097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:48.194022 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:18:48.198268 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:48.198248 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vt95p_3655e63d-2e5a-408d-8af6-b1b312194629/ovn-acl-logging/0.log" Jul 08 18:18:49.842793 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.842747 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8"] Jul 08 18:18:49.849283 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.849254 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:49.856628 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.856587 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8"] Jul 08 18:18:49.984368 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.984321 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44v5s\" (UniqueName: \"kubernetes.io/projected/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-kube-api-access-44v5s\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:49.984558 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.984415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-sys\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:49.984641 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.984565 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-podres\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:49.984641 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.984631 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-proc\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:49.984760 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:49.984663 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-lib-modules\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.085886 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.085848 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-sys\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.085928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-podres\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.085962 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-proc\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.085977 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-sys\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.085988 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-lib-modules\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.086069 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-proc\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086097 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.086086 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44v5s\" (UniqueName: \"kubernetes.io/projected/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-kube-api-access-44v5s\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086334 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.086101 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-lib-modules\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.086334 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.086088 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-podres\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.095426 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.095354 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44v5s\" (UniqueName: \"kubernetes.io/projected/4cc7a847-9ddd-4e05-804d-04d28b31dd3d-kube-api-access-44v5s\") pod \"perf-node-gather-daemonset-7vgq8\" (UID: \"4cc7a847-9ddd-4e05-804d-04d28b31dd3d\") " pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.162205 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.161915 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:50.337953 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:50.337922 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8"] Jul 08 18:18:51.137872 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:51.137807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" event={"ID":"4cc7a847-9ddd-4e05-804d-04d28b31dd3d","Type":"ContainerStarted","Data":"449d234c46fbaf77e41ea19228744d69078dd4c0e437c48f106468567d750bca"} Jul 08 18:18:51.138333 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:51.137903 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" event={"ID":"4cc7a847-9ddd-4e05-804d-04d28b31dd3d","Type":"ContainerStarted","Data":"c0b6cad95e336af657ba641859d72f962b49b35433b4b6f5abf83921aea0fc9b"} Jul 08 18:18:51.138333 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:51.138309 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:51.157026 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:51.156958 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" podStartSLOduration=2.156909979 podStartE2EDuration="2.156909979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:51.155558909 +0000 UTC m=+2703.655338634" watchObservedRunningTime="2026-07-08 18:18:51.156909979 +0000 UTC m=+2703.656689755" Jul 08 18:18:52.082914 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:52.082866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dpxz2_e7c9fd72-b1d2-42d5-9374-018fcd1e6776/dns/0.log" Jul 08 18:18:52.108099 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:52.108062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dpxz2_e7c9fd72-b1d2-42d5-9374-018fcd1e6776/kube-rbac-proxy/0.log" Jul 08 18:18:52.299876 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:52.299850 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zv4d4_3b713581-56c8-46be-bc7c-51279b58448c/dns-node-resolver/0.log" Jul 08 18:18:52.835457 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:52.835418 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b4456856f-c7mmm_5e0109f3-0220-4fda-a6d8-622efc6c99a1/registry/0.log" Jul 08 18:18:52.906812 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:52.906783 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s5tj7_744b8ce9-1887-478f-a460-b648e27cc580/node-ca/0.log" Jul 08 18:18:54.058151 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:54.058108 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7brb2_2a66b758-ef3f-4e43-81d2-0d4807741157/discovery/0.log" Jul 08 18:18:54.213463 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:54.213426 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gfqvm_f0abacd3-e2e7-4f66-a201-3bcf597e469b/payload-pre-processing/0.log" Jul 08 18:18:54.273789 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:54.273757 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dpdct_49d6102f-58cd-4cc4-befc-a8ddeb457b5f/payload-processing/0.log" Jul 08 18:18:55.030200 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:55.030169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zdwng_dc5f6ad9-9612-4a84-a375-36b8f3916d69/serve-healthcheck-canary/0.log" Jul 08 18:18:55.518311 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:55.518279 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nbcq5_508d9609-ed36-456c-9743-8919d19f667d/kube-rbac-proxy/0.log" Jul 08 18:18:55.540506 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:55.540478 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nbcq5_508d9609-ed36-456c-9743-8919d19f667d/exporter/0.log" Jul 08 18:18:55.566741 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:55.566711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nbcq5_508d9609-ed36-456c-9743-8919d19f667d/extractor/0.log" Jul 08 18:18:57.155500 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:57.155467 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cq44k/perf-node-gather-daemonset-7vgq8" Jul 08 18:18:58.292693 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:58.292650 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c548d5447-9s8jh_483dba4c-3621-4126-ae6f-738ee2d29df1/manager/0.log" Jul 08 18:18:58.319310 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:58.319211 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c548d5447-9s8jh_483dba4c-3621-4126-ae6f-738ee2d29df1/copy-manifests/0.log" Jul 08 18:18:59.737964 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:18:59.737933 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8d6bb58cc-t9lph_f13f4971-4cd8-45d8-a7cf-9d56f8d3a1bc/manager/0.log" Jul 08 18:19:00.275700 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:00.275614 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7brb2_2a66b758-ef3f-4e43-81d2-0d4807741157/discovery/0.log" Jul 08 18:19:00.384465 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:00.384434 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gfqvm_f0abacd3-e2e7-4f66-a201-3bcf597e469b/payload-pre-processing/0.log" Jul 08 18:19:00.431117 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:00.431078 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dpdct_49d6102f-58cd-4cc4-befc-a8ddeb457b5f/payload-processing/0.log" Jul 08 18:19:01.743586 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:01.743486 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7brb2_2a66b758-ef3f-4e43-81d2-0d4807741157/discovery/0.log" Jul 08 18:19:01.863091 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:01.863061 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-gfqvm_f0abacd3-e2e7-4f66-a201-3bcf597e469b/payload-pre-processing/0.log" Jul 08 18:19:01.916466 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:01.916432 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dpdct_49d6102f-58cd-4cc4-befc-a8ddeb457b5f/payload-processing/0.log" Jul 08 18:19:06.431567 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.431534 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/kube-multus-additional-cni-plugins/0.log" Jul 08 18:19:06.461612 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.461583 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/egress-router-binary-copy/0.log" Jul 08 18:19:06.484509 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.484485 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/cni-plugins/0.log" Jul 08 18:19:06.507350 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.507323 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/bond-cni-plugin/0.log" Jul 08 18:19:06.530543 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.530509 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/routeoverride-cni/0.log" Jul 08 18:19:06.552194 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.552165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/whereabouts-cni-bincopy/0.log" Jul 08 18:19:06.573223 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.573192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7cfqv_0ecc90ad-bc13-4f07-9e16-a25dd51d9331/whereabouts-cni/0.log" Jul 08 18:19:06.968162 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:06.968123 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nh469_064068a9-6858-474d-a457-b2bc3fb6c435/kube-multus/0.log" Jul 08 18:19:07.070846 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:07.070782 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jwzwj_b815e6c6-8eeb-4e56-9a5d-87b762aae43a/network-metrics-daemon/0.log" Jul 08 18:19:07.093627 ip-10-0-128-115 kubenswrapper[2574]: I0708 18:19:07.093592 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jwzwj_b815e6c6-8eeb-4e56-9a5d-87b762aae43a/kube-rbac-proxy/0.log"